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Abstract 

We evaluate the asymptotics of equivocations, their exponents as well as their second-order coding rates under 
various Renyi information measures. Specifically, we consider the effect of applying a hash function on a source and 
we quantify the level of non-uniformity and dependence of the compressed source from another correlated source 
when the number of copies of the sources is large. Unlike previous works that use Shannon information measures 
to quantify randomness, information or uniformity, we define our security measures in terms of a more general 
class of information measures—the Renyi information measures and their Gallager-type counterparts. A special 
case of these Renyi information measure is the class of Shannon information measures. We prove tight asymptotic 
results for the security measures and their exponential rates of decay. We also prove bounds on the second-order 
asymptotics and show that these bounds match when the magnitudes of the second-order coding rates are large. We 
do so by establishing new classes non-asymptotic bounds on the equivocation and evaluating these bounds using 
various probabilistic limit theorems asymptotically. 

Index Terms 

Information-theoretic security. Equivocation, Conditional Renyi entropies, Renyi divergence, Sibson’s mutual 
information, Arimoto’s mutual information. Error exponents, Secrecy Exponents, Second-order coding rates 


I. Introduction 

Consider the situation where we are given n independent and identically distributed (i.i.d.) copies of a joint 
source ( A n ,E n ). One of the central tasks in information-theoretic security is to understand the effect of applying 
a hash function 0 (binning operator) / on A n . This hash function is used to ensure that the compressed source 
f(A n ) is almost uniform on its alphabet and also almost independent of another discrete memoryless source E n . 
Mathematically, we want to understand the deviation of f(A n ) G {1..... } from the uniform distribution on 

the same support P m i x .f(A r ') and the level of remaining dependence between f(A n ) and a correlated source E". 
These two criteria can be described by equivocation measures. Traditionally in information-theoretic security |2j, 
|3j, equivocation is measured in terms of the Shannon-type quantities such as the Shannon entropy, relative entropy 
(Kullback-Leibler divergence), and mutual information. In particular, it is common to design / such that the 
following is small for any rate R: 

D(P f{A ^ E 4P mixJ{An) x P En ) = nR — H(f(A n )\E n ). (1) 

Clearly if the above quantity is small in some sense, the message f(A n ) is close to uniform and almost independent 
of E n , two desirable traits of a hash function for security applications. 

A. Motivations 

A novel feature of this paper is that we depart from using Shannon information measures to quantify randomness 
and independence. It is known that the Shannon entropy H or the relative entropy D are special cases of a larger 
family of information measures known as Renyi information measures, denoted as H\ +s and I)\ +s for s G M. 
Thus, as expounded by Iwamoto and Shikata [4J, we can quantify equivocation using these measures, gaining 
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deeper insights into the fundamental limits of information leakage under the effect of hash functions. There may 
also be a possibility of the optimal key generation rate changing when we use alternative information measures. In 
addition, in the study of cryptography and quantum key distribution (QKD), the Renyi entropy of order 2 @ (or 
collision entropy) H 2 {A\P a ) := -log J2 a eA P A( a ) 2 and the min-entropy H mhl (A\P A ) := -logma x aeA P A (a) 
play important roles in quantifying randomness. A case in point is the leftover hash lemma §-®. Another 
motivation stems from the recent study of overcoming weak expectations by Dodis and Yu |9j where cryptographic 
primitives are based on weak secrets, in which the only information about the secret is some fraction of min-entropy. 
The authors in [9] provided bounds on the weak expectation E f(Y) of some function / of a random variable Y in 
terms of the min-entropy and the Renyi entropy of order 2. In a follow-on paper by Yao and Li 1101, this study was 
generalized to Renyi entropies of general orders. Finally, in the study of secure authentication codes (or A-codes 
in short), which is one of the most fundamental cryptographic protocols in information-theoretic cryptography, 
Shikata GD quantified lengths of secret keys in terms of Renyi entropies of general orders. Motivated by these 
studies, the authors opine that it is of interest to study the performance of hashing under these generalized families 
of entropies (generalized uncertainty measures) and divergences (generalized distance measures). 


B. Main Contributions 


We consider three asymptotic settings—the asymptotics of Renyi-type security measures, its exponential decay 
and a certain second-order behavior. 

1) First, we characterize the asymptotic behavior of the security measure 


Dl+s(Pf(A"-),E™\\Pmi X ,f(AA X Peu ) ( 2 ) 

for a fixed rate R = d log||/|| where ||/|| := \f{A n )\ is the cardinality of the range of a hash function /. 
The function / will be taken to be a random hash function as we will explain and motivate later. Further, 


(particularly in Corollary [T] therein), we show that if we measure security using D\ +s with s > 0, the 
fundamental limits of key generation rates change relative to those for traditional Shannon-type measures 
D i. The security measure in ([2]) quantifies the deviation of the hashed or compressed random variable f(A n ) 
from the uniform distribution and also its remaining dependence from a correlated random variable E n . 

2) We are also interested in the speed of the exponential decay of ([2]) given a fixed rate II. That is, we are 
interested in the asymptotic behavior of 


as we shall see in Section II-B[ the quantity in (|2j) is closely related to the equivocation [T2J. In Section IF 


-lOg D 1+s (P fiAn)}E r, 


|-fmix,/(.4") X Pe ?l ). 


( 3 ) 


This is likened to error exponent or reliability function analysis in classical information theory 1131, 141. We 
study this in Section fV] 


3) Finally, in Section [V 


we also study the second-order asymptotics 1151, [[161 of the decay of D\ +s with the 


blocklength, i.e., the asymptotic behavior of 


l|-fmix,/(yt") X Pe™)i 

\ogD 1+s {P f{An)iE 4P^ f{M) x P E "). 


and 


( 4 ) 

( 5 ) 


where the number of compressed symbols (size of the hash function) ||/|| equals e nR f '■ j/nL for some first- 
order rate R (usually the conditional Renyi entropy) and second-order rate L 6 K. For some cases (Renyi 
parameter less than one) where we cannot exactly determine the tight second-order asymptotics (i.e., the 
upper and lower bounds do not match), we study the asymptotic behavior of ([2]) when the second-order rate 
L tends to +oo or — oo. In this case, the upper and lower bounds match up to and including a term quadratic 
in L. 


As we mentioned earlier, we will regard / as a random hash function in the sequel. That is, it is randomly 
selected depending on a random variable X n E X n that is available to all parties and is also independent of all 
other random variables. This random variable has distribution Px n - See Fig. [T] To further elaborate, instead of the 
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A n - E n X n ~P Xn 

fx n (') 

fx n (A n )-~- E n X n 

I_I 

Fig. 1. Illustration of applying a hash function fx„ on the source A n . Common randomness X n , independent of a correlated source E n , 
is available to all parties and it determines the hash function fx n - We would like fx n (A n ) to be uniform on its support {1,..., ||/x„ ||} 
and almost independent of E n in the sense of ensuring that quantity in 0 is small. We examine (2) under different asymptotic regimes 
such as the equivocation, the exponential behavior 0. and the second-order asymptotics 0-0- 


the Renyi divergences in Q-Q, for the purposes of asserting the existence of a particular function / with some 
desired properties (cf. the random selection argument), we consider the quantity 

D l+s ■= Di +s (Pf Xn ^),E",xJ\Piaix,f Xn (A n ) x P E n X Px J- (6) 

Here, we note that f Xn is a random hash function (to be defined precisely in Definition [I]) and | f\ n {A") | is a 
constant random variable, i.e., it does not depend on the realization of X n . Even though in (6) is not an 

expectation of any quantity of interest, exp ((1 + s)Z)[“ is the expectation of 

Di2 s (x n ) := exp ((1 + s)D l+s {P f ^ M ) iE „\\P mi ^ fx ^ A , ) x P E -)), (7) 


where the probability of observing x n is P Xn (x n ). Thus by a random selection argument, if the former is less than 
e > 0, there exist an x* n € X n , indexing a deterministic protocol f x ~ , such that 77^(x*) is also less than e. When 
s = 0, the expectation of quantities in Q-Q under the common randomness X n generating a universal hash 
function f Xr , (■) is equivalent to the quantity in ([6]) but for s ^ 0, they are, in general, different. In the sequel, we 
adopt the latter criterion in ([6]) to simplify the presentation of the results. 

We believe the results contained herein may serve as logical starting points to derive tight exponential error bounds 
and second-order coding rates for the wiretap channel 112] (as was done in (T7J, jT8| ) and other information-theoretic 
security problems such as the secret key agreement [ 191 (as was done in (20) , (21) ) problem. The leakage rates for 
these problems may be measured using traditional Shannon information measures or Renyi information measures 
(or their Gallager-type counterparts). Here, we are only concerned with the secrecy requirement rather than both 
the secrecy and reliability requirements of the wiretap problem. The reliability requirement can be handled using, 
by now, standard error exponent analyses [13), 141. 


C. Related Works 


In (Eg, (22), Hayashi generalized and strengthened the seminal privacy amplification analyses of Bennett et al. 
(7), Renner (23) and Renner and Wolf (24) to obtain exponential error bounds for the leakage rate of the discrete 
memoryless wiretap channel and the secrey key agreement problems |2), [3), 1121, [19|. The leakage rate was 
measured by the mutual information I (A RE\Pae) and the variational (or trace) distance \\Pae — Pa x P E \\\- The 


exact exponents for the variational distance are, by now, well known [ 181, [251. 

However, the results concerning the exponential decay of the leakage rate quantified via the mutual information 
contained in GZJ’ and further generalized to other setings in [[20), pT) , are only achievability results (i.e., lower 
bounds on the exponents). The converse has been open for some time. The present contribution, though not focusing 
on the wiretap channel or any specific information-theoretic security problem, derives tight exponential bounds for 
a generalization of the Shannon information measures, namely the family of Renyi information measures. In the 
process, we obtain a tight result for the exponential leakage rate for the mutual information, thus resolving the 
converse part that was open in fl7 ], [20], [21]. As a by-product, for some range of the Renyi parameter, we also 
obtain tight exponents for security measures defined using the Renyi divergence under various hash functions. 

Hayashi and Tsurumaru (26) proposed an efficient construction of hash functions for the purpose of privacy 
amplification with less random seeds, thus potentially realizing the system in Fig. |T] with less random resources. 
Other works along the lines of deriving exponential error bounds for information-theoretic security problems include 
those by Hou and Kramer (27) , [281, Pierrot and Bloch 1291, Bloch and Laneman [30], Han et al. [31] and Parizi 
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and Telatar (32J. After the present work was submitted, Parizi, Telatar and Merhav |33j proved ensemble tight 
exponential error bounds for the wiretap channel by appealing to type counting methods and channel resolvability 
arguments. 


D. Paper Organization 

The rest of the paper is organized as follows: In Section [II] we state the relevant preliminaries and define 
relevant information measures and security criteria for understanding the rest of the paper. In Section [III] we 
state our results for the asymptotics of the equivocation. In Section |W} we state our results for the exponential 
behavior of the Renyi-type security criteria. In Section [V] we state our results for the second-order asymptotics 
of the equivocation. We also consider the case where the magnitudes of the second-order rates are large. These 
are proved using novel one-shot bounds which are stated in Section VI The proofs of the asymptotic results are 
provided in Section VII We conclude the paper in Section VIII| by summarizing our key contributions and stating 
avenues for further investigations. The proofs of the one-shot bounds are rather technical and are thus relegated to 
the Appendices. 


II. Preliminaries and Information Measures 


A. Basic Shannon and Renyi Information Quantities 

We now introduce some information measures that generalize Shannon’s information measures. Fix a normalized 
distribution Pa £ 'PiA) and a non-negative measure (a non-negative vector but not necessarily summing to one) 
Qa £ P iA) supported on a finite set A. Then the relative entropy and the Renyi divergence of order 1 + s are 
respectively defined as 


D(Pa\\Qa) :=^fii(a) log 

aeA 


fit (a) 
Qa(o) 


D 1+s (P a \\Qa) := - log y2P A (a) 1+s Q A (a)- s , 

S A 

aeA 


(8) 

(9) 


where throughout, log is to the natural base e. It is known that lim s _m Di +s (P a \\Qa) = D(P a \\Qa) so a special 
case of the Renyi divergence is the usual relative entropy. It is known that the map s i-a sDi +s (Pa\\Qa) is 
concave in s £ M and hence Di +s (P a \\Qa) is monotonically increasing for s £ M. Furthermore, the following 
data processing or information processing inequalities for Renyi divergences hold for s 


D(P a W\\QaW) < D(Pa\\Qa) 
D 1+s (P a W\\QaW) < D 1+s (P a \\Qa). 


( 10 ) 

( 11 ) 


Flere W : A —> B is any stochastic matrix (channel) and P A W(b) := )P n XT'"(61a.)fi \(r/) is the output distribution 
induced by W and P A . 

We use finix,A to denote the uniform distribution on A. We also introduce conditional entropies on the joint 
alphabet A x £. If Pae is a distribution on dxf, the conditional entropy and the conditional Renyi entropy of 
order 1 + s relative to another normalized distribution Qe on £ as 


H(A\E\P ae \\Qe) := -D(P AE \\I A x Q e ), (12) 

H 1+s {A\E\P ae \\Qe) := -D 1+s (Pae\\Ia X Qe). (13) 

Flere I a {cl) = 1 for each a £ A and it is known that lim s _>.o Hi+s{A\E\P A e\\Qe) = H{A\E\P AE \\Qe)- If 
Qe = P’e. we simplify the notation and denote the conditional entropy and the conditional Renyi entropy of order 
1 + s as 

H(A\E\P ae ) := H(A\E\P AE \\Pe) = ~ J2 P e^) E P A\E(a\e) log finale) (14) 

e a 

H 1+s (A\E\Pae) := H 1+s (A\E\P A e\\Pe) = -~ s logJ2 p E(e)J2 P A\E(a\e) 1+s - (15) 
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The function s i->- sH\ +s (A\E\P ae ) is concave, and H\ +s {A\E\P ae \\Q E ) is monotonically decreasing on (0, oo) 
and (—oo,0). 

We are also interested in the so-called Gallager form of the conditional Renyi entropy for a joint distribution 

Pae € V(A x £): 


hI +s (A\E\P A e ) := -^logj] ( 


\l+s 


By defining the familiar Gallager function d, [34] (parametrized slightly differently) 


f(s\A\E\P AE ) := log EE Pae(cl, e) 1 ~ 


1 —s 


we can express © as 


h\ +s {A\E\P ae ) = 


1 + S 


A\E\Pae), 


(16) 


(17) 


(18) 


thus (loosely) justifying the nomenclature “Gallager form” of the conditional Renyi entropy in ( fT6| ). The quantities 
H i+ s and H\ +s can be shown to be related as follows: 


max H 1+s (A\E\P ae \\Q e ) = h\ +s {A\E\P AE ) 

Qe€ r\t) 


(19) 


for s G [—l,oo) \ {0}. The maximum on the left-hand-side is attained for the tilted distribution 


QE(e) 


E a P^(q,e) 1+ ‘)^ _ 

Ee(Ea P AE{a,ey+ s )^ 


( 20 ) 


The map s -A sE[\ +s (A\E\P A e) is concave and the map s i— > h[ +s {A\E\P A e) is monotonically decreasing for 
s G (—l,oo). It can be shown by L’Hopital’s rule that 


lim hI +s (A\E\P AE ) = H(A\E\P ae ). 


( 21 ) 


Thus, we regard H[{A\E\P A e) as H(A\E\P ae ), i.e., for Renyi parameter a = 1 + s = 1, the conditional Renyi 
entropy and its Gallager form coincide. We also find it useful to consider a two-parameter family of the conditional 
Renyi entropy: 

1 

H l+s \ 1+t {A\E\P AE ) := log J2 P E(e) ( J2 ^A|E(a|e) 1+s ) 1+ ‘. (22) 

e ^ a ' 

Clearly, 

H 1+s1i+s (A\E\ p ae) = h\ +s (A\E\P ae ) (23) 


so two-parameter conditional Renyi entropy is a generalization of the Gallager form of the conditional Renyi entropy 
in ( fl6l ). 

For a fixed joint source P AE define 


R s := j t tH 1+t {A\E\P AE ) 
Rt := ±tHl +t (A\E\P AE ) 


and 


(24) 

(25) 


We note that R s and /(I are monotonically non-increasing in s because the functions t tHi +t (A\E\P AE ) and t 
tR[\ +t (A\E\P AE ) are concave. The fact that t ^ —tPl\ +t (A\E\P AE ) is convex is because the maximum of convex 
functions is convex; cf. m Furthermore, both R s and III are non-negative by direct evaluation of the derivatives 
and noting that log P A \ E {a\e) < 0. We assume, henceforth, that the source P AE satisfies the conditions that 
t tHi + t(A\E\P AE ) and t ^ tPl\ +t (A\E\P AE ) are both strictly concave so R s and III are both monotonically 
decreasing in s. 
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The Renyi entropies can be shown to satisfy a form of data processing inequality. In particular if / : A —> M. 
is any function on the set A, we have 

H(f(A)\E\P AE ) < H(A\E\P ae ), (26) 

H 1+s (f(A)\E\P AE ) < H 1 +s (A\E\P ae ), (27) 

H\ +s (f{A)\E\P AE ) < h\ +s {A\E\P ae ). (28) 

Inequalities ( [27] ) and ( |28| ) hold true for all s > —1. These inequalities say that processing the random variable A 
cannot increase its randomness measured under any of the above conditional Renyi entropies. 


B. Renyi Security Criteria 

Now, we introduce various criteria that measure independence and uniformity jointly. The mutual information is 


I(A A E\P ae ) := D(P AE \\P A x P E ). 


(29) 


This, together with its normalized version, has been traditionally used as measure of dependence in classical 
information-theoretic security |2J, |3!j, going back to the seminal work of Wyner [12] for the wiretap channel. It 
was also used by Ahlswede and Csiszar for the secret key agreement problem 1191. However, it does not guarantee 
approximate uniformity of the source P A on A. Thus, we introduce the modified mutual information 


C(A\E\P ae ) -.= D(P AE \\P rmX;A X P E ) (30) 

= \og\A\-H(A\E\P AE ). (31) 

This quantity was also considered by Csiszar and Narayan [ [35] Eq. (6)] in their work on secrecy capacities. An 

axiomatic justification of C(A\E\P ae ) was provided recently by Hayashi [36, Thm. 8]. The modified mutual 
information C(A\E\P ae ) clearly satisfies 

C(A\E\P ae ) = I (A A E\Pae ) + D(P A \\P mixA ). (32) 

Hence, if C(A\E\P ae ) is small, A is approximately independent of E and A is approximately uniform on its 
alphabet, desirable properties in information-theoretic security. We may further generalize the modified mutual 
information by considering Renyi information measures, introduced in Section |II-A[ as follows: 

Ci +s (A\E\P ae ) ■= D 1 +s {P A e\\P^,a x Pe) (33) 

= \og\A\-H l+s {A\E\P AE ). (34) 


This can be relaxed to give yet another security measure—the Gallager-form of the modified mutual information: 

C\ +s {A\E\P ae ) := min Di +s (P AE \\P m ix,A x Q e ) (35) 

Qe€'P(£) 

= \og\A\-H\ +s (A\E\P AE ). (36) 

We characterize these quantities asymptotically when (A, E) = (f(A n ).E n ) for some (classes of) hash functions 
/(•). The quantities H\ +s and II j +s can be regarded as equivocations [12] so Cj +iS and C]^ s are the negative of 
the equivocations up to a shift. We work with C\ +s and Cj , , in the rest of the paper as they are more convenient 
and they admit the interpretation as security criteria. 


C. Decomposition of the Renyi Security Criteria into Mutual Information and Divergence Terms 

We note that for any s > — 1, C\ +s (A\E\P ae ) = 0 if and only if P AE = P mix . A x P E or equivalently, A is 
uniform on A and statistically independent of E. This is because I)\ + fiP\\Q) is a divergence so D\ +S (P\\Q) = 0 
if and only if P = Q [37]. The same is true for the case C\ +s (A\E\P ae ) = 0. From this observation, we see that 
Cj +S (A\E\ P/\e) and C[ +s (A\E\P ae ) also measure how close the source or “key” A is to uniform and how secure 
A is from an adversary E. Thus the quantities we consider are generalizations of the standard security measure 
C(A\E\P ae ) in (|32|) and measure uniformity and security in a different way. 
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More quantitatively, one may wonder whether the security criteria C\ +S (A\E\P AE ) and C\ +s (A\E\P AE ) ad¬ 
mit a decomposition into “mutual information” and “divergence” terms and similar to (321. We first consider 
C\-\- s (A\E\Pae)- Define g s {a) '■= E e Pae{o,, e) 1+s P E (e)~ s . We then see from the definition of the Renyi diver¬ 
gence of order (1 + s) that 


^sDi + s (Pae 11 ^Pmix, .A XP.e) 


(w 

= (r 
= (r 

As a result, one has 

Dl +s (PAE\\Pmi*,A X Pe) = 


Y ( ^ p AE{a,e) 1+s P E (ey 


• *—a / i \ i+ s 

Y [9s( a ) 1+5 ) 


EsR')+ + 'E 

a' a 


9s(a) 1+s 
Za'9s(a')^ 


1+s 


(37) 

(38) 

(39) 


1 + s 


log Y 9s (a') + + - log Y 


9 s(a) 


-E a' 9+0 1+s 

Invoking the definition of g s (a ), we see that the first term can be rewritten as 

1 S \—a , i 1 -|- S 


1+s 


+ log|A|. (40) 


log^S+O 1 ^ 


log Y Pa{o!) ( Y p E\A{e\a') 1+s P E (ey 
a 1 ' e 

=: /{+ son) (.B A A\Pae)- 


(41) 

(42) 


This is exactly Sibson’s definition of the order-(l + s) Renyi mutual information [38]. See Verdu’s work in [39 

A\P AE 

r (Sibson) 


Sec. Ill] for the properties of J-[+ son (T? A A\Pae) and a generalization to arbitrary alphabets. See Hayashi’s 


work [40 


Sec. II.C] for a generalization of / 


(E A A\Pae) to quantum systems. The work of Tomamichel 
and Hayashi in ||4T| Sec. IV.B] provides an operational interpretation of this quantity in the context of composite 


1+S 


hypothesis testing. The sum of the second and third terms in (40) form a Renyi divergence of order (1 + s). 
In particular, the second term is the negative Renyi entropy of order (1 + s) of the probability mass function 

Qa } (°) := ffs(a) + /Ea'2s(a / ) I + ; * Hence, 


D 1+s (P AE \\P mix , A x P E ) = /(^ son) (E A A\Pae) + Di +S (Q^ ||Pmi x ,A) • 


Mi 


(43) 


Because = P A , and lim s _>.o (EaA\P A e) = I{A/\E\P AE ), the decomposition in (43»is a generalization 

of (|32|). Equation ([43]) is also reminiscent of an information geometric Pythagorean theorem [42| (but for Renyi 


(s) 

divergence here). The distribution Q y A x P E can be regarded as the 17i + . s -information projection of Pae onto the 


set {Qa x Pe ■ Qa £ P(A) }. This was observed in the quantum information context by Sharma and Warsi [43 
Lemma 3 in Suppl. Mat.]. They called the relation the quantum Sibson identity. 


Next we consider the Gallager-form of the modified mutual information C\ +s {A\E\P AE )- From (36), it can be 
seen by adding and subtracting H\ +s (A\P a ) that 


C\ +s {A\E\P ae ) = H 1 +s (A\Pa ) - H[ +s {A\E\P ae ) + D l+S (P A \\P^ A ). 


t 


(44) 


We recognize that the sum of the first two terms constitutes Arimoto’s [44] definition of the order-(l + s) Renyi 
mutual information 


I 


(Arimoto) 

1+s 


(A A E\Pae) = H 1 +s (A\Pa) - h\ +s {A\E\P ae ). 


(45) 


Since hm^o l{+ imot +A A E\P ae ) = 7(A A E\P AE ), the security criterion C\ +s (A\E\P AE ) also admits a 

Sec. II.A] for detailed discussions of the properties of /|^ l s " riot °* (A/\E\ Pae)- 


decomposition similar to ([32]). See [39 

























Fig. 2. Schematic showing the relation between the various entropies and the transition rate R- s (defined in and (|511>). The figure 
with the Gallager forms of the conditional Renyi entropy H\ ±a and ff_ s (defined in (|25|) and (|52|>) is completely analogous. See Fig. 4 


III. Asymptotics of the Equivocation 

In this section we present our results concerning the asymptotic behavior of the equivocation. First we define 
precisely the notion of hash function. This is a generalization of the definition by Carter and Wegman QJ. 

Definition 1. A randon^] hash function f x is a stochastic map from A to M := {1 ,M}, where X denotes a 
random variable describing its stochastic behavior. An ensemble of random hash functions fx is called an e-almost 
universal hash function if it satisfies the following condition: For any distinct a \. a2 G A, 


Pi' (/x(ai) = fx{a 2 )) < 

When e = 1, we simply say that the ensemble of functions is a universal hash function. 


(46) 


As an example, if we randomly and uniformly assign each element of a E A into one of M bins indexed by 
m E M. (i.e., the familiar random binning process introduced by Cover [451), then Pr(/y(«i) = fx («i)) = jj so 
this is a universal hash function, and furthermore, ( |46| ) is achieved with equality. 

Let |f| + = maxjd, t). The following is our first main result. 

Theorem 1 (Asymptotics of the Equivocation). Le^M n = e nR . Assume that f \ n ■ A n — > M. n = {1,..., M n } is 
a random hash function^ For any s 6 [0,1], we have 


lim 1 2 inf C l+s {fxM n )\E n X n \Pl E x P X J = | R- H 1 +s (A\E\P ae )\+, 

n—tco n fx„ 

lim - mf Ci +s (fxM n )\E n X n \n E xP x J = \R~ h\ +s {A\E\P ae )\ + . 

n—>oc n f Xn 


(47) 

(48) 


Furthermore, for any s <G (0,1], we also have 


lim - inf Ci- s {f Xri {A n )\E n Xn\P AE X P x J = 

n—too n fx n 

lim - inf C\_ {f Xri {A n )\E n X n \P AE x P x J = 

n — ^00 77- fx n 


R-H^ s (A\E\P ae ) R>R-s 

maXj 6 [ 0)S ] l(R - Hi_ t (A\E\P AE )) R < R_ s 


R - H[_ s (a\e\p ae ) 


R> R 


—s 

it 


max te[M 1 ( R ~ H^^fiAlElP^)) R < Rf 


(49) 

(50) 


1 For brevity, we will sometimes omit the qualifier “random”. It is understood, henceforth, that all so-mentioned hash functions are random 
hash functions. 

2 As is usual in information theory, we ignore the integer effects on the size of the hash function M n = ||/|| since this is inconsequential 
asymptotically. This imprecision is also employed in the sequel for notational convenience. 

Tn particular, all the infima in \A1\ (| 50 1 1 (as well as similar statements in the sequel) are taken over all fx n that are random hash functions. 
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Fig. 3. Illustration of the security measures Ci+ s and C\- a (for s £ [0,1]) in ( |47| i and (49} for the discrete memoryless multiple source 
Pae where Pae(0, 0) = 0.7 and Pae( 0,1) = Pab(1,0) = Pae( 1,1) = 0.1. 



Fig. 4. Illustration of the security measures C\ +3 and C|_ s (for s £ [0,1]) in (48} and (50} respectively for the same source. 


where recall that R_ s and R}_ s are defined in © and ( [25] ) respectively. (Also see © and (52} for alternative 
representations.) Furthermore, the infima in (47}—(50} are achieved by any sequence of e-almost universal 2 hash 
functions fx„ (where e is a fixed positive number). 


This result is proved in Section VII-A The ideas to prove the direct parts (upper bounds on the leakage rates) 
for the s = 0 cases are contained in previous works such as I JT7), ©, ©. All other parts are novel. 

We remark that the converse parts (lower bounds) to (47}-(48} hold for all s > 0 (and not only being upper 

. Furthermore, instead of the formulae in (24} 


bounded by 1) owing to the data processing inequalities in (27} 

and (25}, the rates in which the behavior of the security measures change R_ s and Rj_ s can also be expressed as 


R- s = ^ tF[ 1 - t (A\E\P AE ) 
R t _ s = ^tHl_ t (A\E\P AE ) 


and 


t=S 


t=S 


(51) 

(52) 


These alternative expressions for R- s and R 


will be useful in the proof of Theorem [I] 


The results in (47}-(5Q} imply that an optimum sequence of hash functions { fx„ jnei! is such that asymptotically 
the normalized security measure C\ +s and its Gallager-type counterpart C\ +s increase linearly with the rate R if 
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the rate is larger than the conditional Renyi entropy and its Gallager-type counterpart. However, note that this only 


holds for the case where R is greater than the analogue of the critical rates, defined in (|24[>—(|25 [) in the case where 
the Renyi parameter a = 1 — s is less than one. Observe that there is difference in behavior when we consider the 
other direction, i.e., the quantities Cj and C[_ s for s E [0,1]. Below the critical rate, the equivocation no longer 
increases linearly with R but is nonetheless still convex in R. See Fig. [2] for a schematic of the various rates and 
the behavior of the equivocations. We numerically calculate the asymptotics of the equivocations in Theorem [T] 
and display the results in Figs. [3] and [4j The behaviors of the normalized security measure C\ +S ,C\- S and their 
Gallager-type counterparts Cj, s ,Cj_ s are similar. 

Finally, we examine the optimal (maximum) key generation rates, i.e., the largest rates R for which there exists a 
sequence of functions from A n to {1,... , e ni? } such that T(2 1+s or ^C\ +s tend to zero as the blocklength grows. 
We observe from the following corollary that this cutoff rate depends strongly on the sign of s. In particular for 
s E (0,1], the cutoff rates are Hi +s (A\E\P AE ) and H[ +s {A\E\P ae ) respectively, while for s E [—1,0], the cutoff 
rates are both equal to the Shannon conditional entropy H(A\E\P AE ) independent of s. This difference between 
the behaviors of the optimal key generation rates depending on the sign of s (also illustrated in Figs. [3] and [4]) is 
somewhat surprising (at least to the authors). 


Corollary 1 (Optimal key generation rates). We have 

Ci+s (/ (A n ) | E n | P AE ) 


sup < R E R+ : lim inf 

n—>°o >{!,...,e nJl } 


= 0 ^ = 


n 


Cl +s (f(A n )\E n \P2 E ) 


sup < R E R+ : lim inf 


= 0 ^ = 


n 


H 1+s (A\E\P ae ) 

if s€ (0, 1 ] 

, (53) 

h(a\e\p ae ) 

if s e [- 1 , 0 ] 

h\ +s (A\E\P ae ) 

if s e ( 0 , 1 ] 

• (54) 

H(A\E\P ae ) 

if s E [-1,0] 


Proof: We only prove the statement for C\ +s in ( |53| ) since that for C[ +a in ( |54| ) is completely analogous. The 
case for s E (0,1] is obvious from ( [47] ) in Theorem TjTsince the limit is \R — Hi +s (A\E\P ae )\ + . Now, for the 
case s E [—1,0], if R < H(A\E\Pae), we know from the monotonically decreasing nature of H\ +s {A\E\P ae ) 
(in s ) that R — Hi_t(A\E\P AE ) is non-positive for t E [0, s]. Thus, referring to ( [49] ) in Theorem [I] the optimal t 
in the optimization max te [ 0 s ] [ (R — H\ _ t (A\E\ P AE )) is attained at t = 0 and consequently, the optimal objective 
value is 0. On the other hand, for any R > H(A\E\P ae ), the optimal t E (0, s] and so the optimal objective 
value is (strictly) positive. Thus, for s E [—1,0], the optimal key generation rate is the Shannon conditional entropy 
H{A\E\P ae ). This concludes the proof for ( |53] ). ■ 

In Section |I-A we alluded to the importance of the collision entropy H 2 in cryptography and QKD. The 
implication of (531 in Corollary [T] is that if we operate at a hashing rate II > H) and we employ the security 
criterion C 2 , then there will inevitably be some residual leakage of the source A n given a hashed version f(A n ) 
and side-information E n . 


Because of the normalizations of C\ +s and C\ +s by n in ( [53] ) and ( |54| ), Corollary [I] is analogous to results in the 


vast majority of the literature in information-theoretic security |2j, ||3] where the weak secrecy criterion is employed. 
We address the analogue of the strong secrecy criterion 1461 in Theorem [2] to follow where we not only demand 
that the unnormalized quantities Cj + , s and C]_ s vanish with n, we also demand that they do so exponentially fast 
and we identify the exponents. 


IV. Exponential Behavior of the Security Measures 

In this section, we evaluate the exponential rates of decay of the security measures C\± s and C'] ±s for fixed 
rates R above an analogue of the critical rate. 


Theorem 2 (Exponents of the Equivocation). Let M n = e nR . Assume that f x„ '■ A" —t M. n = {1,.... M n } is a 
random hash function. For R > R 1 (R s being defined in (|24[) ), and any s E [0,1], we have 


lim -- log inf C 1+s (fxAA n )\E n X n \P 7 f E x P Xn ) 

rwoo n fx n 

lim —— log inf C\- s {fx n {A n )\E n X n \P r f E x P X J 

n—>oo n fx n 


sup tHi +t {A\E\P AE ) - tR 

te[s,i) 

max tHi +t (A\E\PA E ) — tR. 
iG [0,1] 


(55) 

(56) 
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Rate R 

Fig. 5. Illustration of the exponents of the security measures Ci+ a and Ci- a (for s £ [0,1]) in (55} and ( [56^ respectively for the 
discrete memoryless multiple source Pae as in Fig. [3] The curves for C 1/2 and C\ are identical and they are equal to zero for all rates 
R > Hi = 0.4400 bits per source symbol (cf. Corollary ]T}. 


For the Gallager-type counterparts of the Renyi quantities and R > R[ (Rh being defined in (25 \), and any 
s G [0,1], we also have 


lim - 1 log inf Cl +s (f xn {A n )\E n X n \Pl E x P X J = 
n—too n fx 

1 


max tHi +t (A\E\PAE) ~ tR 
te[s,i] 


lim — log inf C {_ s (f Xn (A n )\E n X n \P2 E x P x J = max tH l+t (A\E\P AE ) - tR. 

n.-too n fx n ie[0,l] 


(57) 

(58) 


The infima in 


are achieved by any sequence of e-almost universal 2 hash functions f x „. 


This result is proved in Section |VII-B| The techniques for the direct parts are somewhat similar to those in 
(T7), (2TJ, |22) using improved versions of Bennett et al.’s [7J bound which was based on the Renyi entropy of 

order 2. However, the non-asymptotic bounds (e.g., Lemma [5]) and asymptotic evaluations for the converse parts 

_ s 

require new ideas. Different from the direct part, we need to convert the evaluation of e ' 1 and e l ~> 1-0 into 


information spectrum [47] quantities (involving the conditional entropy random variable) so that is is amenable to 


asymptotic evaluation. These information spectrum quantities are then evaluated using various large deviation [481 
bounds, such as Cramer’s theorem. We make several other observations. 

First, the exponents of the security indices (namely — ^logCi-ts and —7 log C'J ±S ) are non-negative because 
C\± s < log|/x„ (A n )\ = 0(n) and C\± 8 < log | f x „ (A n ) | = 0(n) (cf. their definitions in (34]) and (36])). The 
and 


expressions in 


are already nonnegative and so we only need to include the 


operation for (551 


and (57 1 . 


Second, the derivative of the conditional Renyi entropies R\ and ll\ are the analogues of the critical rate in 


error exponent analysis [131, [ 14]. For the exponents, we have a complete characterization of the exponential rates 
of decay of both C\± s and for s G [0,1] and they are given by optimization of quantities that are related 
to the conditional Renyi entropy. We observe that the Gallager form results in larger exponents in general as the 
optimizations in (56} and (58} are larger than their non-Gallager counterparts in (55} and (57} respectively. 

Finally, the exponents in (55} and (56} of Theorem [2] are illustrated in Fig. [5] We observe the same behavior for 
the exponents of the Gallager forms in (57} and (58} since the expressions are the same and so we omit these cases. 
We note (from the plot and from direct evaluations) that the zero-crossings for the exponents of C 1 / 2 , C\. C 3/2 and 
C 7/4 occur at // \.H\. H 3 / 2 and /7 7/4 respectively (Hi being the Shannon entropy). This is in line with Corollary [T] 
Indeed, the exponent being positive implies that the normalized security measures -Ci± s and -C\± s vanish as 
blocklength grows. Thus, we conclude that the optimal key generation rates under both the strong and weak secrecy 
criteria are the same. 
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V. Second-Order Asymptotics 


In the previous sections, the security measures in terms of equivocations and their logarithms were normalized 
by the blocklength n. In this section, we study different normalizations, e.g., by y/n. In addition, we examine the 
effect of changing the size of the hash function M n from e nR (considered in Sections 
where L e M is an arbitrary real number. 


Ill 


and 


IV) to e nR +V™ L f 


A. Basic Definitions 

To present our results, we first define the following important quantities. 


Definition 2. Given a discrete joint source P AE G V(A x £), define the conditional varentropy 1491 or conditional 
source dispersion /50/. [fiTj to be 

, 2 


V(A\E\P ae ) :=Y / PAE(a,e)(logP A]E (a\e) + H(A\E\P AE )y. 


a,e 


We also define the following variants of the conditioned varentropy 

Vi(A\E\P ae ) ■,= Y, p E(e){H(A\E\P AE ) - H(A\P A]E=e )) 2 

e 

V 2 (A\E\P ae ) := V(A\E\P ae ) - V\{A\E\P ae ) 

= ^2, p ae{0”, e) (log P A \ E (a\e) + H{A\P A \ E=e ))' 

a,e 


(59) 


(60) 

(61) 

(62) 


One can readily check that V = V\ + V 2 from the definitions. This also follows immediately from the law of 
total variance. Let 

1 ft 

$(i):=-=/ e-“ 2/2 d u (63) 

V 27T J —00 

be the cumulative distribution function of the standard Gaussian random variable. With these definitions, we are 
ready to state our results on the second-order asymptotics for the security measures C\ +s and G') +iS which are simple 
functions of the equivocation H\ +s and H\ , s respectively. Note that the second-order analysis of C\ (corresponding 


to the s = 0 case) with no side information (i.e., E = 0) was performed in Hayashi’s work [16 Theorem 8] in 


the context of intrinsic randomness based on the relative entropy (Kullback-Leibler divergence) criterion. The other 
results in Theorems [3] and [4] are novel. 

To state our result succinctly, we define the quantities which all depend on s, L and P AE (but we suppress the 
dependence on the fixed joint distribution P AE for brevity): 


r l(s ,L) := A log (2, i. s , !,(!_„)) - 

r 2 (s, L) — lo «*( - ,JViA\E\P AK ])' 

'Ll (s,L) := -log f2 s+T37 s T3 » (1 — s 

s V 


(64) 

(65) 


1 — s 


log 


$ - 


L + x 


4 


tb p -x 2 /(2V 1 (A\E\P ae )) 


'T 2 (s,L) := - 1 —- log $ ( - 


L 


y/V(A\E\P AE ) J ' 


yV 2 (A\E\P AE ) J ^eVMIEIPae) 


dx, 


( 66 ) 

(67) 


B. Bounds on the Second-Order Asymptotics 

Theorem 3 (Second-Order Asymptotics). Assume that fx TL : A n -A M. n = {1..... M n } is a random hash function. 
Consider the following three cases: 
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Case (A): a = 1 + s with s G (0,1]: Suppose that the number of messages M n = e nH i+A A \ E \P A E)+fnL or 
M n = e n - H Us(A\E\P AE )+fnL_ when L>0, we have 


lim — inf C l+s {fxM n )\E n X n \Pl E x P x J = L 
n-> oo Jn f : Y „ 


( 68 ) 

(69) 

Similarly to Theorem [7] the infima in (681 and (69 1 are achieved by any sequence of e-almost universal hash 
functions f x „ ■ 

When L < 0, we have 


lim -= inf C\ +s [fxM n )\V n X n \P n AE x P x J = L. 

n^-oo /x„ 


lim -4= lo S inf ^i+-(/x„(A n )| J B n X n |P2 E x P X J = -sL 

n-s-oo -i/n / x „ 


lim —^ log inf Ci +s (fxM n )\E n X n \Pl E * PxJ = sL. 

n^oo Wn fx„ 


(70) 

(71) 

Similarly to Theorem [2] ?/;<? infima in © and 4ZD are achieved by any sequence of e-almost universal 2 hash 
functions fx.„ ■ 

• Case (B): a = 1 (i.e., s = 0 ): Suppose that M n = e nH (A\E\p AE )+\/nL y or .some L G R, we have 

lim -L inf C' 1 (/x n (^ n )|^ n X n |Pl E x P Xn ) = f L/VX {AlElPAE) L ~VV(A\E\PA E)x e - x >/2 dx (72) 

n^oo ^/n /x„ J.oo V2vr 

the same asymptotic behavior also holds truefor the Gallager version of the security measure 
C](fx.„ (A n )\E"X n |Pf [7 X Px,,)- Similarly to Theorem I /I the infima in ( |72| ) is achieved by any sequence of 
e-almost universal hash functions f x „ ■ 

• Case (C): a = 1 — s with s G (0,1]: Suppose that M n = e n H{A\E\P AE )+y/nL y or some L G K, we have 


max{ri(s,L),r 2 (s,L)} < lim inf inf Ci^ s (f Xn (A n )\E n X n \P2 E x P X J 

n—¥oo /x„ 

< limsup inf Ci_ s (/ x „(A n )|B; n X n |P 2 E x P Y J < ^ (S,L) . 

n —>00 Jx n 1 5 


(73) 


7/7 addition, for the Gallager-type counterparts, with M n = e nH (A\E\P AE )+fn.L L G R, we flZ.ro / 7 flve 

max{’I'i(s,L),^2(s,B)} 

< lim inf inf Cl_ s (f x „ (A n )\E n X n \P% E x P Xn ) 

n^oo f Xn 

A 


< limsup inf C[„ s (fxM n )\E n Xn\Pl E x P x J 

n—>■ 00 Jx n 

h(s,L) + ~ log (V + ^s^(l - s)) . 


< T'i 


(74) 


r/ie upper bounds in (731 and m are achieved by any sequence of e-almost universal 2 has functions f x „ ■ 

This result is proved in Section |VII-C| We remark that the converse parts (lower bounds) to ([68])—((69]) hold for 
all s > 0 (and not only being upper bounded by 1) owing to the data processing inequalities in (|27[)—(|28]) 


C. Remarks on Theorem [7] 

Observe that in Theorem [3] (Case (A) for instance), the number of compressed symbols M n satisfies 

\ogM n = nH\+ s (A\E\P ae) + y/nL, or, (75) 

log M n = nH\ +s {A\E\P AE ) + y/nL. (76) 

The leading conditional Renyi entropy terms scaling in n are known as the first-order terms, while the terms scaling 
as y/n are known as the second-order terms. The coefficient L is known as the second-order coding rate [16l], 
{52] , [53] and the second-order asymptotic characterizations depend on L. Note that even though L is termed as 
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Fig. 6. Illustration of the second-order asymptotics in Case (B) given by the right-hand-side of \12\ for the discrete memoryless multiple 
source Pae as in Fig. [3] It is easy to see that the integral there is non-negative. 


the second-order coding rate, it may be negative. Observe that the conditional varentropies appear in (|72l)-(|74|). 
which suggests that we evaluate the one-shot bounds using the central limit theorem among other techniques. We 
have tight results (equalities) for Cases (A) and (B) but unfortunately not for Case (C) where the Renyi parameter 
a = 1 — s for s E (0,1]. However, in the limit of the second-order coding rate L being large (either in the positive 
or negative direction), we can assert that one of the terms in the maxima in the lower bounds of ( |73] ) and ( |74| ) 
dominates and matches the upper bound and hence, we have a tight result up to the term in L 2 (Theorem [ 4 ]). We 
now comment specifically on each of the cases. 

1) For Case (A), the second-order asymptotic behaviors of C\ +s and Cj_ 5 when they are normalized by -^= 
are linear in L. 

2) The same is true for Case (B) for large positive L because with V := V(A\E\Pae), 


lim T / L/ ^ L 7^V^ d , = 

L->oo L J_ 00 y/2lT 




rL/W xe -x 2 /2 


V2tt 


dx > = 1. 


(77) 


3) 


In contrast, when L —> —00 in Case (B), the limit is zero. The second-order asymptotics in Case (B) in ([72]) 
is shown in Fig. [6] and is obtained via numerical integration to approximate the integral. The limit in ( |72| ) is 
monotonically increasing in L. This is intuitive because as L increases, there is potentially more leakage to 
E n and less uniformity on the (larger) support {1,..., e n.ii(A\E\p AH ) \ fhLj 

For Case (C) there is no normalization by and we only have bounds. However, for large \L\, we will see 
from Theorem [4] that the second-order asymptotic behavior is quadratic in L. The bounds on the second-order 
asymptotics in the two parts (conditional Renyi entropy and its Gallager version) of Case (C) in ( |73| ) and ( |74| ) 
are shown in Figs. [7] and [8] respectively. 


We conclude that in the second-order asymptotic regime where the number of compressed symbols satisfies ([75])- 
there are distinct differences between the three regimes of the Renyi parameter a E [0,1), a = 1 and 

a E (1, 2], 


D. Approximations for Large Second-Order Coding Rates 

Since for Case (C) we only have bounds, we now examine the behavior of the bounds in the limit of large \L\ 
for which we can show tight results up to the quadratic terms. 

Theorem 4 (Large Second-Order Rates). Assume that fx n ■ A n -P AT,, = {1,..., M n } is a random hash function. 
For Case (C) in Theorem [7] (Renyi parameter a = 1 — s where s E (0,1]), we have the following asymptotic results 
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Negative L Positive L 




Fig. 7. Illustration of the bounds on the second-order asymptotics of Ci/ 2 (/x n (A n )\E n X n \Pf E x Px„) (i.e., s = 1/2) in Case (C) given 
by the left- and right-hand-sides of {73} for the same source. Note that the figure on the left is plotted in log scale (corresponding to {80}) 
while the figure on the right is plotted in linear scale. For L < 0, lower bound 1 in ( |73|> i s negative (lower bound 2 dominates) so is not 
shown in the left plot. Observe the quadratic behaviors; this is corroborated by Theorem H] The quadratic approximations in {78} and {80} 
(without the O(logL) terms) are also plotted. Observe that there is a constant offset between the quadratic and the bounds as we do not 
determine the O(logL) terms in Theorem [4] exactly. 


as L 


oo: 


lim inf C\- s (fx n {A n )\E n X n 

n^oo f Xn 

lim miCi_ s (fx n (A n )\E n X n 

n^oo f Xn 


Pae X Px J = 


P%E X Px,,) = 


l? 


2sV{A\E\P ae ) 
1 — s 


+ 0(log L), 
L 2 


2s V 1 (A\E\P ae ) + V 2 (A\E\P ae )(1-s] 
Furthermore, we have the following asymptotic results as L -A —oo: 

L 2 


(78) 


+ 0(logL). (79) 


lim log inf Ci- s (fx n (A n )\E n X r> 

n^oc f Xn 

lim log inf Ci_ s (f Xn (A n )\P n Xn 

n—>°o f Xn 


Pae x Px,,) = - 


I P n 
\ r AE 


X Px ,,) = - 


2V(A\E\P ae ) 

L 2 


+ 0(log|L|), 


+ 0(log \L\). 


(80) 


(81) 


2V(A\E\P ae ) 

The infima in {78}-{8l} are achieved by any sequence of e-almost universal 2 hash functions fx„ ■ 

The proof of Theorem |4 can be found in Section |V-E 

The results in Theorem 4] are somewhat analogous and similar those in the study of the moderate-deviations 
asymptotics in information theory [54|-p8|. Note the difference between the results in {78}-([79l> (L -a 00 ) versus 
(80 i-([8 T|) (L -A — 00 ). The former pair of results resembles the equivocation results presented in Section III since 
the effective rate is (L / \/n) -higher than the conditional Renyi entropy and there is no logarithm preceding C) _, s 

since the effective rate is (\L\/y/n)- 


IV 


and Cj_ s . The latter pair of results resembles the exponent results of Section 
lower than the conditional Renyi entropy and there is a logarithm preceding C\ _. s and C[_ s . So the results presented 
in Theorem [4] are natural in view of the equivocation result in Theorem [T] and the exponent result in Theorem [2] 

E. Proof of Theorem [7] 

In this section, we present the proof of Theorem [4] Since there are four statements in (f78T)-([8T]). we partition the 
proof into four distinct subsections. 
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Negative L Positive L 




Fig. 8. Illustration of the bounds on the second-order asymptotics of Cf 2 (fx n (A n )\E n X n \P A E x Px„ ) (i.e., s = 1/2) in Case (C) given 
by the left- and right-hand-sides of 0 for the same source. For L < 0. lower bound 1 in is negative (lower bound 2 dominates) so 
is not shown in the left plot. Observe the quadratic behaviors-this is corroborated by ( |79| > and (|8 1 [i in Theorem [4] These plots are obtained 
by using numerical integration to calculate the integral in '1 /1 (s,L) in f/4}. 


a) Proof of ( f78] ).- When L -> oo, the term ri(s, L) in ((64]) behaves as — Mog < h( — 


+ 0 ( 1 ), 


y/V{A\E\P AE )' 

and attains the maximum in the lower bound in ( [73] ) because ro(s,L) has the additional factor 1 — s, which is 
smaller than 1. Also see the right plot of Fig. [7] Thus, in this limiting regime, the lower bound matches the upper 
bound in ( [73] ), namely — llog<I?( — -/=====), up to a constant term, i.e., 

lim inf C^ s (f Xn (A n )\E n X n \P2 E x P Xn ) = --log( --= f ) +0(1) 
rW00 f Xn s \ yfV (A\E\P AE ) ) 

where 0(1) denotes a term bounded in L (but dependent on s). Now by employing the asymptotic equality 

e-F/2 


(82) 


4>(-i) = 1 - 3>(i) ~ 


y/2nt ’ 


we obtain from ( |82| ) that 

lim inf C^ s {f Xn {A n )\E n X n \Pl E x P x J = 


fx n 


as t —> oo, 


L 2 

2sV{A\E\P ae ) 


+ O(logL), 


(83) 


(84) 


which proves ([78]). 


b) Proof of (79): When L 


oo. 


the term T'i (s,L) dominates the maximum in the lower bound in ( [74] ) 


because V 2 < V and thus the integrands in +1 (.s, L), namely <h( — 


L-j-x 


;) 


e -^/(2V 1 (AIEIP AE )) 


are not 


smaller than <!>( — 


y/V 2 (A\E\P AE )> y /2nV 1 {A\E\P AE ) 

, ---- ) in + 2 ( 6 , L). See right plot of Fig. 8 We can then find the x that dominates the 

-+V( j 4|E|P.4e) / _|_| 


integral in +i(s,L). We denote this by x*. Since L is large, by 


log 


$ - 


L + x\ 1 ~ B 

~vwj 


x 2 /(2V 1 ) 


Differentiating the quadratic, we obtain 


x = — 


1 /L + x\ 2 

' 2(1 -s) WWJ 

LV\(A\E\P ae ) 


x 

2V\ 


0, as L —> 00 . 


Vi(A\E\P A e) + (1 - s)V 2 (A\E\P ae ) ■ 


(85) 


( 86 ) 
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The exponential term e ( x *) 2 /( 2V i(A\e\p ae )) controls the behavior of the integral in T i (s, L) and substituting ( | 86 | ) 
into this exponential term yields < f79l >. 

c) Proof of ( [80] ) : Now we assume that L —t — oo. In this case, we find that the term T 2 (s,L) attains the 
maximum in the lower bound in ([73]) because Ti(s, L) is negative due to the constant negative term. Also see the 
left plot of Fig. [7] In this case, taking the logarithm, we have 


lim log inf C^sifxM^E'X^PlE x P X J 

n^oo f Xn 


= log 
= log 


— log 1 — $ 


L 




V V(A\E\P ae )J 


+ 0 ( 1 ) 


$ 




L 


{^V(A\E\P ae )) 
L 2 


+ 0 (log|L|) 


2V(A\E\P ae ) 


+ 0 (log|L|) 


(87) 

( 88 ) 

(89) 


where in log(l^) and log(^) can be regarded as 0(1) when L 


—oo. 


in ( 88 ), we used the fact that 


log(l —t) = —t + 0(t 2 ) when t f 0, and finally in ( |89] >, we used (83). This proves ( |80| ). 

d) Proof of ( | 8 T] ).' In the other direction, when L —t — oo, we claim that the term ^(s, L) attains the maximum. 
This is shown as follows: First, we find that 


1 - 


L + x 


K^V 2 (A\E\P AE y) 


>1 - J-gf L+x 

1-s \^V 2 (A\E\P ae ) 


(90) 


This is because ana 1 - is convex and so the linear approximation underestimates the function. This means that 


log / $( - 

J — OO 
> log 

= log 1 - 


L + x 


d 


iTl p —x 2 /(2 Vi(A\E\Pae)) 


^V 2 (A\E\P ae ) J y/2 +V^A\E\pXe) 


dx 


1 - 
1 


1 L+x \ 

1-s \y/V 2 (A\E\P AE )) 


e ~x 2 /(2V 1 (A\E\P AE )) 

^2ttVi(A\E\P ae ) 


dx 


-<f> 


L 




i-s-\^/v(a\e\p ae )J 


M 

i 


_ L _^ + O ( e -E/AV(A\E\P AE ) 2 )\ 

v V(A\E\P AE )J V J 


1 — s 

1 

T+ 


log 


1 - $ 


L 


V v(a\e\p ae )J 


+ O (e~ Li /( 4V l A \ E \P A *) 2 )) 


— log< p(- , L ^ + O ( e -P/AV(A\E\p AE ) 2 )\ 

-s \ yfV(A\E\P AE )J V J 


(91) 

(92) 

(93) 

(94) 

(95) 


where ( |92| ) follows because the convolution of two independent zero-mean Gaussians is a Gaussian where the 
variances add and we also note that V = V) + If per (| 6 T|). This argument was also used in the second-order analysis 


of channels with state [59 Lemma 18]. Inequalities ([93]) and (94) follow from the fact that log(l — x) = —x+0(x 2 ) 


as x | 0 (note that L -A — oo so the term +(L/\/V) tends to zero). Hence, ([95]) and the definitions of L) 


and T7 1 ( Sj L) (in ( 66 ) and (67 1 resp.) imply that + 2 (s,L) asymptotically dominates T' 1 (s,L) as L — > — 00 . Also 
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see left plot of Fig. [8] By a similar calculation as in ([87])— (f89|), we obtain the lower bound to ( |8T| ) as follows: 


limMlogmf Cl_ s (fxAA n )\E n X n \P2 E x P Xn 

n^-oo f Xn 


> log 
= log 
= log 


'1 

1 — s 
s 

1 — s 
s 


log 4> ^ — 
(-$( 


L 


^V(A\E\P ae )) 

L \ 


$ 


L 


\^v{a\e\p ae )J 

^ + 0(log \L\) 


+ 0 ( 1 ) 

+ 0 (log|L|)) 


^V(A\E\P ae )J 

L2 + 0 (log|L|), 


2V(A\E\Pae) 


(96) 

(97) 

(98) 

(99) 


where in (96[ ), we used the above observation that ^ 2 {s,L) = f2(^ r i(s,L)) as L —> —oo, in (97] ), we wrote 
<&{—L/s/V) = 1 — §(L/y/V) and used the fact that log(l — x) = — x + 0(x 2 ) as x f 0, in ( [98] ) we used the fact 
that log[(l — s)/s] = 0(1), and finally in (99] > we used the approximation in ( |83| ). To show that the upper bound in 
(74[ ) matches the lower bound given by (99[ ) (when L -» —oo), we use (90) and steps similar to those in ([97])—([95]) 
to assert that 


1 — s 


log 


$ - 


L + x 




x 2 /(2V 1 (A\E\P ae )) 


y/V 2 (A\E\P AE )J y/ 2 nVi(A\E\P AE ) 


dx 


1 — s , 

<-log 


= 

s 


1 - 
L 


L 




l-s \^V{A\E\P ae )J _ 

t ^ + O L-Pl^v{A\E\p AB y)\ 
V V(A\E\P ae )J V ) 


( 100 ) 


( 101 ) 


where in ( 101 1 , we again used the fact that log(l — x) = —x + ()(x 2 ) for x i 0. Now taking the logarithm and the 
limit as L -» —oo, we match the lower bound in ([99]) completing the proof of ( [ 8 T] ). 

VI. One-Shot Bounds 

To prove Theorems [TJ [2] and [3] we leverage the following one-shot (i.e., blocklength n equal to 1) bounds. The 
proofs of these one-shot bounds are rather technical and hence we provide them in the appendices. 


A. One-Shot Bounds for the Direct Parts 

For the direct parts of the equivocation results, we evaluate the following one-shot bounds. The first two bounds 
in (102) and ( 103 ) can be considered as generalizations of the bounds by Hayashi in (17]] where e = 1. 


Lemma 1. For an ensemble of e-almost universal hash functions fx : A -X A4 = {1 we have for 

s G [ 0 , 1 ], 

e sC 1+s (f x (A)\EX\P AB xP x ) < e s + M a e -aHi+,(A\E\P AK ) 

0 ^ c l + .(MA)\ EX \ p AB X p x ) < +M jf ae -j^Hl + M\E\P AB )' 

In the other direction with s £ [0,1], 

Q—sC 1 _ s (f x (A)\EX\P AB xP x )) 

>2~ s J2 PAE(a,e)P AlE (a\e)- s M- s + 2~ s PA E {a,e)e~ s , 

(a,e):P A |E(a|e)>^ (a,e):P A | B (a|e)< ] | 

e -TE a Ct_ s (fx(A)\EX\P AB xP x )) 

>-+7rE F ®to( E PaiMF-)^ +&r*'£ P *:(e)( E PA\E(a\e))^. 

e a:P A \ B (a\e)>-^ e a:P AlB (a\e)<^ 


( 102 ) 

(103) 


(104) 


( 105 ) 
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For the direct parts of the exponents results, we evaluate the following one-shot bound. 

Lemma 2. For an ensemble of universal hash functions fx ■ A — > FA = {1, ..., M}, we have for any s G [0,1], 

e ff c i+s(fx( A )\EX\P AE xP x ) < ]^ _|_ ^ ^s c ~sH 1+s (A\E\P ae )' (106) 

_ 1 + s 

For the direct parts of the second-order results, we evaluate the following one-shot bound. 

Lemma 3. For an ensemble of an e-almost universal 2 hash functions fx '■ A —> A4 = {1 , ,M}, we have for 
any s G [0,1] and c > 0, 

e -.C,..(/,M)|EA|F„xP,) > p 4E { (o , e) . p A]Me) < (107) 


,-fr 3 CLAfx(A)\EX\P AE xP x ) 




e)f p A\E(a\e] 

a:P A \ B {a\e)< 


B. One-Shot Bounds for the Converse Parts 

For the converse parts of the equivocation results, we evaluate the following one-shot bounds. 
Lemma 4 . Fix c > 1 and s > 0. Any hash function f : A —t A4 = {1 ,,M} satisfies 
e -sC,-Af{A)\E\p AE ) < c -s^p E (e) J2 PA\E(a\eY~ s M- s 


a:P A]B {a\e)>^ 


+ 2i-. a i-.(l - s)PAe{(a,e) : P A \ E {a\e) < -^}. 


For the Gallager-type counterpart, 


e -^cLM(A)\ElP AB ) < 2T f, Y^PE(e) (c~ s J2 PA\E{a\ef- s M- s y 

e a:P A]B (a\e)>-^ 

+ f2 l -=s 1 -»(l — s)P A \ E=e {a : P A \ E (a\e) < — 


+ (1 - s)P A \ E=e ^a : P A \ E (a\e) < -jj }) • (HO) 

For the converse parts of the exponents results and the second-order results for the Renyi parameter being 1 + s 
(with s nonnegative), we evaluate the following one-shot bounds. 

Lemma 5. Fix c > 1 and s G [0,1]. Any hash function f : A —> M = {1,..., M} satisfies 
esCi-Af{A)\E\p AB ) < P E (e)P AlE (a\eY- s M- s + ]T P E (e)P A \ E=e {a : P A \ E (a\e) < 

( a,e):P A]E {a\e)>^ e 

( 111 ) 

< PA,E\[a,e) : P A \ E {a\e)> YY\ C ~ S + P A , E \[a,e) \ P A \ E {a\e) < YY\ • ( 112 ) 

For the Gallager-type counterpart, for s G [0,1), 

e -*cr..(/(A„F|P„) < J2r E (Y p ^A a: > A}c- 

e L 

1 "1 1 

+ PA\E=e{a ■ PA\E(a \ e ) < Jf] ■ ( 113 ) 

In the other direction, for s G [0,1], we have 

e sC 1+s (f(A)\E\p AE ) > ^ P E (e)P A]E (a\e) 1+s M s 

{a,e):P A \ B {a\e)>-^ 

+ Y,Pde)P A \E=e{(a,e) : P A \ E (a\e) < ^} 1+ * (114) 

e 

> P A E\{a,e) \ P A \ E {a\e)> YY^ 3 + P AE \{a,e) : P A \ E (a\e) < Y-Yf ■ (H5) 
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For the Gallager-type counterpart, for s <G [0,1], we have 


iT ^cl +e (f(A)\E\p AE ) > ^2 P E (e) 


Pa\E=e\ a '■ PA\E( a \ e ) > 


+ Pa\e=e 


{a : P A \E( a \e) < 


Mi 

l+s' 


-1 

Mi 


(116) 


For the converse parts of the second-order results for the Renyi parameter being 1 — s (with s nonnegative), 
we need the following one-shot bound as well as ( | 110 [ ) and ( ] 11 3| ) although the converse parts of the second-order 
results with Renyi parameter being l + s require d 1 14| ) and ( 1 16[ ). 

Lemma 6 . Fix c > 1 and s G [0,1]. Any hash function f: A —? A4 = {1,..., M} satisfies 

e - sCl -sU(A)\E\P AB ) < c -sp AE |( a e) . p A ^ a | e) > 


-1 

Mi 


+ 2 1 --’s 1 - 


Pae{(o a e) : P A \ E (a\e) 


<-}■ 
~ Mi 


(117) 


VII. Proofs of the Asymptotic Results 

In this section, we prove the asymptotic results in Theorems [T] [2] and [3] 

Notation: Throughout, we let a = (ai, 02 ,..., a n ) G A n and e = (ei, e 2 ,..., e n ) G £ n denote deterministic 
length-?! strings. We also let A n = (Ai, A 2 , ..., A n ) and E" = (E\, E 2 , ..., E n ) denote random vectors of length 
n. We adopt the exponential equality notation: a n = b n if and only if linq,^^ - log jf = 0. 

Given a random variable X with distribution (probability mass function) P, we denote the expectation of a 
function of the random variable g(X) by E[fj(Xy = V r I’fcjgfi:). If we want to make the dependence of the 
expectation on X or P explicit, we write Ex[g(2f)] or Ep[g(X)]. The same comment applies to the variance 
operator which we denote interchangeably as Var[g(Jf)], Var^ [fj(X j] or Var p[g(X)\. 


A. Proof of Theorem [7] 

1) Direct Parts: We first prove the direct parts (upper bounds). 

a) Proof of the upper bound of ( |47j ).- The bound in d 1 02[ ) implies that 


C 1+s {fxM n )\P n Xn\Pl E X PxJ) 


< J log (e s + M*e- sHl +°( An 1 ^”!^)) 

(118) 

= J log ( e s + M^e~ nsHl +^ A \ E \ p ^i). 

(119) 


For e being a constant, this achieves the upper bound of ( |47| ) upon normalizing by n and taking the lim sup. 
b) Proof of the upper bound of ( |48| ): The bound in ( |103[ ) implies that 

Cl +s (fxM n )\P n Xn\Pl E X P X J) 


< 


l+s, r ~nr, -S- 


■ log (e 1 +’ + M n 1+ 


e n-' 


h 1 +s {AAeEPa E ) 


) 


= log (ei+ + Mpe-^n^^ Pae) 


)■ 


( 120 ) 

( 121 ) 


This leads to the upper bound of ( |48| ) for constant e upon normalizing by n and taking the lim sup. 

c) Proof of the upper bound of ( |49l ): To obtain ( [49] ), we employ Cramer’s theorem [481 on the sequence of 
random variables — logP^ E {A n \E n ) = 1 ~ 1°S PA\E(At\Pi) under the product joint distribution F’fp. It is 

easy to see by using exponential tail bounds that 


lim-log 

n—>00 Ti 


E 


Pae ( a ’ e ) 


(a,e):P" |E (a|e)<e- 


1 f 1 71 

= lim-log Pae\ (a,e) : - V log P A \ E ( 

00 n n zJ 

l 2—1 

= ma xt(R - Hi- t (A\E\P ae))■ 


< -R 


( 122 ) 

(123) 
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Note that the cumulant generating function of the random variable — log P A \ e (A\E) under the joint distribution 
P AE can be expressed in terms of the conditional Renyi entropy as 


ioglEp^ 


g t(— log Pa\e(A\E)) 


tH^ t (A\E\P AE ), 


(124) 


explaining the presence of this term in d 1 23[ ). We again apply (a generalized version of) Cramer’s thcorcnj^] to 
the sequence of random variables log P^, E (A n \E n ) under the sub-distribution (non-negative product measure) 
P2 E (a,e)(P^"|g) ri (a|e) and event {(a,e) : logP^| E (a|e) > —nR}. Note that the cumulant generating function in 
this case is 


r s (t) :=log'^2P A E(a,e)P A * E (a\e)ex.p(tlogP A \ E (a\e)) (125) 

a,e 

= (s-t)H 1 _ {s _ t) (A\E\P AE ), (126) 

and by direct differentiation, we also have that 

r'(0) = -R- s (127) 

where R s is defined in ( |24| ) (cf. R- s is presented in a different form in (|5T])). Thus, by Cramer’s theorem, 

jgJ e il°gPA| E (A|_E)j 


E PJ B (a,e)P; |E {a|e)- = 

(a,e):P" E (a|e)> e -™* 


a —tnR 


= exp 


—n —tit — 


tR — log E Pae(cl, e)P A \ E (a\e) s e 


t log Pa\e(A\E) 


= exp [-n (-tR + {t- s)H 1+ ( t _ s) (A\E\P AE ))] 


(128) 

(129) 

(130) 


where in d 1 28| ), the “expectation” E is taken with respect to the non-negative measure (a, e) i—)■ P AE (a, e)P 4 |p(a|e). 
Since t > 0 is arbitrary, 


-log 

n 


pae (^ e )-PAip( a i e ) 


(a,e):P^| E (a| e )>e _T *- R 

= max { - tR+ (t - s)H 1+ ( t _ s )(A\E\P AE )} 


(131) 


For the case where R > /i_. s , the constraint in the optimization above is active, i.e., t* = 0 because the function 
s i-A R-s is monotonically non-decreasing as described in Section II-A Conversely, when R < R- s , the constraint 
is inactive, i.e., the maximum is realized with R = R_r s _ t y Thus, we obtain 


1 

lim-log 

n—>• oo Ti 


Y, Pi‘ E (a,e)PJ |E (a|e)-e-” R 

(a,e):P^| E (a| e )> e -"H 

s(R - H^AIEIPae)) if R > Rs 
max t '< s t\R - Hi- V (A\E\P ae )) if R < R- s 


(132) 


where the second clause follows by the substitution t 1 = s — t. Now with these preparations, we can employ the 


one-shot bound in (104 1 with e = 1 to prove the direct part of (491 as follows: Since (132> is not greater than 


4 The standard Cramer's theorem [48 Section 2.2] (or Sanov’s theorem |48[ Section 2.1]) is a large-deviations result concerning the 
exponent of P n (B) where P is a probability measure and B is an event in the sample space fl. If P is not necessarily a probability measure 
but a finite non-negative measure (as it is in our applications), say p, Cramer’s theorem clearly also applies by defining the new probability 
measure B P(B) := p(B)/p(Q). 
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(1231, the former dominates in the exponent and we obtain 


limsup —Ci- s (fx ri {A n )\E n X n \P2E X P Xn ) 

n—>-oo Tl 

1 ,1 


=-lim inf — log 

s n—»oo n 

<-lim inf — log 

s n—>oo n 


-sC 1 - B Ux n (A^)\E^X n \P2 E xP x J 


(133) 


E 


Pae{ & i e ) 


(a,e):P^ |E (a|e)<e-" Ji 


\—s~—snR 


(134) 


+ 2 s ^ P 4 £ (a, e)P^| £ (a|e) s e 

(a,e):P" |E (a|e)>e-'*« 

where ( |134[ ) follows from ( |104[ ). Now we combine the asymptotic results in ( |123[ ) and ( |132[ ) to evaluate the 
asymptotic behavior of ( 134 >. In particular, we take into consideration the scaling factor We also note that the 
domain of maximization of t in d 1 23[ ) and t' in the second clause of d 1 32| ) are [0, oo) and (oo, s] respectively. 
So the intersection of these domains is [0, s] and the eventual max should thus be taken over [0, s]. Uniting these 
observations, we obtain that the upper bound 

1 


lim sup —C±- S {fx n (A n )\E n X n \P2 E x P Xn 

n—>• oo Tl 


< 


(135) 


R-H^AIEIPae) if R>R- S 

max te[0:S ] ±(R - Hi_ t (A\E\P AE )) if R < R- s 

d) Proof of the upper bound of ( |50| ): The upper bound of ( [50] ) proceeds in an analogous manner. It proceeds 
in five distinct steps, each detailed in the following five paragraphs. 

In Step 1, we manipulate the one-shot bound in ( |105[ ) with e = 1 as follows: 

1 


lim sup -Cl_ s (f Xn (A n )\E n X n \P2 E x P x J 

n—>oo Tl 


-lim inf — log 

s n—>oa n 


<-lim inf log 

S n —^oo 


e - T ! r s CLAfxn(A n )\E n X n \PX B xP Xn j 

E 


(136) 


-e-^ nR 

2 


+ 


2e~° 


E p £( 

e 

E F «( e )( E 


P A\E^ 


a|e) 


l-s 


a;p AI E ( a l e )> ee “' 1J? 

P n A\E^f- S 


(137) 


In the following two steps, we evaluate the first and second terms in the lim inf in ( |137[ ). 

In Step 2, we evaluate the second term in the lim inf in ( |137[ ) as it is simpler and provides the intuition and 
techniques for evaluating the first term. For this, we need to employ the Gartner-Ellis theorem | |48| (instead of 
Cramer’s theorem). Doing so to the sequence of random variables — logP^ E {A n \e) = ^ ; " =| — log P A \j7(A t \ei) 
with e of fixed type [14] and A n with the memoryless distribution P'^ F J-\c), as will be shown in the following, 
we obtain 


lim-logY'Pgte 

n—>-oo Tl z — J 

e 

t 


E 

a:P A| E ( a l e )< <:e_ ’ ,H 


= max 


t> o l-s 


-^A|s( a l e ) 

{R- H^^MIEIPae)) , 


(138) 


where H \_ L \ \ _ S {A\E\ Pae) is the two-parameter conditional Renyi entropy defined in (221. To show ( 138 ), consider 
e G Tq = {e G £ n : type(e) = Q}. Then the Gartner-Ellis theorem [48] yields that 


P 


A"|P"=e 


{a : P A \e( 


a e < ee 


—nR 



( 

I = exp 

—n max 
i t> 0 


tR-E Q \o g J2p\t(a\E) 


(139) 
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where 


eq log e p lrJ< a i E > = E «(<=) '°g E p me ( 


a e 


(140) 


Let V n (S) be the set of n-types with alphabet £. Splitting the sum on the left-hand-side in ( 138 1 into the polynomially 
many n-types on £, we obtain 


E p » 


E j’SeMo) 


ee£" a:P" |JS (a|e)<ee-"« 


Y P E(T Q )ex p 

QeP„(£) 


n 


■ max 


1 — s t >o 


tR-E Q log 'Y P A\E^ a \ p ) 


= max exp ( — nD(Q\\PE)) exp I —--max tR — Eq log E p lri<“i E ) 

ri ) \ 0 


= exp —n mm max 
\ Q t> o 


= exp —n max mm 
\ t>o Q 


tR ^og^ a P^(a\E) 


1 — s 


1 — s 


tR E Q logX aJ P^(a|£) 


1 — s 


1 — s 


+ D(Q\\P E ) 


+ D(Q\\P E ) 


(141) 

(142) 

(143) 

(144) 


where ( 142 1 follows from the fact that Pe{Tq) = exp(—n/9(Q|| Pe)) [14, Ch. 2 ], the swapping of min and max 
in ( |144| ) follows from the fact that the objective function is convex and concave in Q and t respectively, Q resides in 
a compact, convex set (the probability simplex) and t resides in a convex set [0, oo) (Sion’s minimax theorem [60]). 
Now by straightforward calculus, the optimizing distribution for fixed t is 

->i -t. 


Q*(e) = 


PE(e)(Z a P$(a le)) 1 - 


Z. 


S,t 


where the normalizing constant (partition function) 


z,,.:=E p ^) E p y< 


a e 


(145) 


(146) 


Plugging this into (144) we obtain 


E p E( e > 

eS £ n 


E 

a:P A|E( a l e )< £e_,lH 


t 


= exp l — nmax 


t>o 1 — s 


R - l -Yz s , t 


(147) 


which then yields ( |138[ ). Note that we have to use the Gartner-Ellis theorem (and not Cramer’s theorem) because 
the collection of random variables {— log P A \ E {-^-i\ e i) : i = 1 ,..., n} is independent but not identically distributed. 

In Step 3, we evaluate the first term in the liminf in ( |137[ ) again by applying the Gartner-Ellis theorem j48| to the 
sequence of random variables log PL R (4"|e) = XILi l°g p A\E{Ai\ e i) with non-negative measure P’t E (j e 


11 —s 


we have 


E P A, e( 

a;P A|Js( a l e )> ee_ '* H 


a e 


, 1 —s 


= exp — n max 
\ t> o 


tR-E Q logY pl A \E 


(148) 


where Q is the type of e and Eq log P \ I e ^ ( a l p ) i s defined in d 140[ ). So by using a type partitioning argument 


of sequences e similarly to d 141 [ )—( p~47] ) , we obtain 


E p £(< 


E 


r>r i ( 1 —s 

r A\E 


(a| e ) 


a:P A|E( a l e )> ee_r ‘' R 


= exp — n max 
\ t> o 


tR 
1 — s 


+ log Y Pe ^)(Y P a\e f) ( a l e 


(149) 
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Consequently, considering the two different cases similarly to (1321, we obtain 


lim-log 

n—>oo Ti 


-nR ' 


£ P% , E (a|e) 


1 —s 


a: - P AIE( a l e )> ee_rlR 


(150) 


f j^ s (R-Hl_ s (A\E\P AE )) ]fR> AtH^^AlElPAE)]^ 

\ max te[o,s] j^(R ~ Ri-t\i-s(A\E\PAE)) A R < ^.tH 1 _ t \i_ s (A\E\PAE)\ t=s 

In Step 4, we put together the asymptotic evaluations in ( |138[ ) and ( |150[ ) in the bound in d 137[ ). We observe 
that (| 1 38[) is not smaller than (|150i. Thus, the former dominates the exponential behavior, and plugging d 138 i into 


(137 1 , we obtain 


lim sup -C\_ s {fxM n )\E n X n \Pl E x P x J 

n—^oo R 


< 


R - hI_ s (A\E\P ae ) if R > £ tHi_ t \i_ s (A\E\P AE ) 

max i£[0|S ] - H^i-^AIEIPae)) if R < gj i-^i-tit-s^l^l-PAB) 

Finally in Step 5, we show that the transition rate in (|151[) 


t=S 

t=S 


^tH 1 _ Al _ s (A\E\P AE )\ t=s = Rl 


d t 


as follows. Since max t H 1 _ s \ 1 _ t (A\E\P AE ) = H 1 _ s \ l _ s {A\E\P AE ) (i.e., the maximum is attained at t = s ), 

^sH^.MIEIPae)^ = s^H^.MIEIPae)^ = 0 . 


dt 

Hence, choosing fi = ^2 = t, we have 

Ri s = ±tHl t (A\E\P AE )\ t=s 


dt 

= ^tH^MWPAE)^ 

= ~^~i^~ tlHl ^d-t2( A \ E \ PAE )\ t =s + i-t 2 ( A \ E \ PAE )\ t=s 

= ^ tH^.MlElPAE)^ + ^ sH^ sll _ t (A\E\P AE )\ t=s 


dt 

d 


— ^ tHi- t \i-s(A\E\P AE )\ t=s , 


(151) 

(152) 

(153) 

(154) 

(155) 

(156) 

(157) 

(158) 


where ( |155[ ) follows from the relation in ((52]), d 1 56| ) follows from the chain rule, and ( |158[ ) follows from d 1 53[ ). 
The relations in ( 151 1 and ( 152[ ) complete the justification of the upper bound of (50). 

2) Converse Parts: For the converse, we do not consider the common randomness X n (i.e., X n = 0) since the 
bound must hold for all (not just e-almost universal) hash functions fx„ ■ This statement applies to the proofs of 
all converse bounds in the sequel. 

a) Proofs of the lower bounds of © and ( |48| ).' The lower bounds to © and ( |48| ) can be easily obtained 
by using the data processing inequalities for Renyi conditional entropies and their Gallager-type counterparts in 

©-©. 

b) Proof of the lower bound of ( |49| ).- Now for ( |49( ), we note that when R > R- s , we have 

s(R - Hi_ s (A\E\P AE )) < max t(R - H^AIE^ae)) (159) 

te[o,s] 


and when R < R- s , equality holds since t = s attains the maximum. Fix t G [0, s]. From ( |109| ), we obtain the 
bound 


^sC 1 . a {f{A-)\E^\P2 B ) < c -«J^p Bn (e) P A\E( a \ e y~ SM i 

e a:P" |JS (a|e)>^|- 


1 —S Ti/T — 8 
n 


+ 2 1 “S 1 8 (1 — s)P AE {( a : e ) : P^ E (a\e) < 

< c -s e sH^ B {A"\ E "\n E )M- s + 2^s^-\l - s)e t H ^-M"\ E AP2 E )(M_Ey\ 


(160) 


(161) 
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where in (jT6Tj). we upper bounded the probability in the second term using Markov’s inequality, i.e., for any 

t £ [0, s], 


P 


’S E {(a,e) : PJ |E ( a|e) < f-} = Pi E {(a.e) : .-‘“W' 1 > 


Ep» 

1 A 


< 


a t log PJI ie (A n | -E n ) 




= e tH^ t (A n \E n \P2 E ) (Mp'j \ 


(162) 

(163) 

(164) 


Put c = 1 in ( | 161 ). We then obtain the lower bound to ( |49| ) by applying ( |159[ ) and its equality version for R < R- s . 
c) Proof of the lower bound of ( f50| ).' Finally, ( [50] ) can be obtained by evaluating ( |1 10[ ) as follows: 

p -fr a CLM(A")\E"\P2 B ) 


< 




P £ .(e)(c- X) P^a^'-M, 

a; - P AiJs( a l e )^iiFr 


l-s i\,r-s \ 1 ~‘ 
n 


< 2~“ 


= 2~“ 


+ ^ 2l " S1 s (1 _ 'S)-PA"|E"=e| a : ^A|£:( a l e ) < fff\j 
' c - s eT pHLfA^\P 2E ) M -YP 

+ 2P^sP^P{l - s)0* ^Pg(e)e^ Hl - t(A " |PA7l|B "=‘ i) (—) _T ^ 

e 

c - e Tfl^(^l E "l^ E ) M -I^ +2 ^ sT lf7 F(1 _ s) T^ e T4 7 H 1 _ t|1 _4^|B"|Pl E )^^-T i 


(165) 


(166) 


(167) 


S s 1 

with s > t > 0. For brevity, let (3 S := 2d-*) 2 s(*- s > 2 (1 — s) 1 -* be a function that only depends on s. By taking the 
logarithm of (| 1 67|), normalizing by n, and using M n = e nR , we obtain 


n 


C\_ s {f {A n )\E n \P 7 f E ) 


. 1 - s 

>-log 


sn 

1 — s 
sn 


log 


2 ^ 


2~s 


c -s eT ^Hl_M"\EAP2v) M -^ + 


c s e -nft-fR-Hl_ B {A\E\P AE )) + ^-n^R-H^^^A^P^)) 


(168) 

(169) 


Now similarly to ( |159[ ), we have 

s(R - h\_ s {A\E\P ae )) < max t(R - H^^AIEIPae)). 

t£[0,s] 


(170) 


Thus when R > R_ s , the first term in (1691 dominates (exponent is not larger), resulting in the first clause 


in ( [50] ). namely R — H[_ s {A\E\P A e)■ On the other hand, when It < R_ s , the second term in ( |169[ ) dominates. To 
complete the argument, we optimize over t £ [0, s] to obtain the second clause in ( [50] ), namely max te [ 0)S ] |(f? — 
Hi_ t \i_ s (A\E\P AE ))- This completes the proof of the lower bound of (50». 


B. Proof of Theorem [2] 
1) Direct Parts: 
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a) Proof of the lower bound of ( [55] ) : We note, per the discussion following Theorem [2] that 


C\ +s {fxM n )\E n X n \Pl E x P x J) = 0(n). 

Thus, the exponents are lower bounded by zero, explaining the 
Now for the non-trivial (non-zero) lower bound on the expoi 
We recall that M„ = e nR . Now we have 


= O(n), and 

(171) 

= O(n). 

(172) 

+ in (|55]) and ([57]). 


s, we employ (| 1 02 ) with e 

= 1 and t G [s, 1]. 

Px, J) 


x PxJ) 

(173) 

?"l P2e)\ 

(174) 


(175) 


< -ntH 1 +t (A\E\P AB ) 

~ t n 

Taking the logarithm and optimizing over t G [s. 1], we obtain the lower bound to 

b) Proof of the lower bound of ([57]) : Similarly, applying (|1Q6[) to the case t G [s, 1], we obtain 


C\ +s UxM n )\E n Xn\Pl E x PyJ) 

< C\ +t {f x M n )\E n X n \Pf E x P Xn )) 

(176) 

< 1 + t logfl+ 1 M^e~ tHl+t ( A "\ En \ p ab ) \ 

~ t V i +t n J 

(177) 

< 1 M* e - ntHl +t( A \ E \ P AB) 

~ t n 

(178) 


which implies the lower bound to ([57]) upon optimizing over t G [s, 1], 


c) Proofs of the lower bounds of ( p6] ) and ([58]) / For the — s versions in (J56J) and ( p8] ), we simply note that 

Ci +S > Ci_ a , (179) 

Cl +8 > Cl s> (180) 


for any s, s' G [0,1] because as mentioned in Section II-A (after ( [15] ) and ( [20] ) respectively), H i +s and h\ +s are 
monotonically decreasing in s. Thus, we have 

liminf -- \ogC^Afx n (A n )\E n X n \Pf E x P x J) >liminf -- log C 1 +s (fxAA n )\E n X n \P% E x P x J) (181) 

n—> oo Tl n—>-oo Tl 

liminf — — log C\_ ,(f x (A n )\E n X n \P^ E x P X J) >liminf--log C\ +s (f Xri (A n )\E n X n \P% E x P x J). (182) 

n—u n—Tl 

Combining these statements with the bounds derived in ( 173 )—( 178] ) completes the proof of (56) and 
2) Converse Parts: 


a) Proof of the upper bound of (|55): For the converse, we first show the upper bound to ( |55] ). Choose a 
constant cq satisfying Cq > 1 + s. Recall the definition of R s in (24). Now assume that R > R s . We claim that 


lim-log 

n—>oo Tl 


P n AE {(a,e) : ^(ale) > coe"^ } cgg-^e^] = sH 1+s (A\E\P ae ) - sR. (183) 
This is justified as follows. We know from Cramer’s theorem [48] that 

Pae {(a,e) : P\ | S (a|e) > c 0 e _n ^| = exp max {tHi +t (A\E\P AE ) - . (184) 


The maximum appeared in right-hand-side of ( |184[ ) is attained when the derivative of tH\ + i(A\E\Pae) — tR s is 
zero because tH\ + t{A\E\PAE) is concave in t. Flence, the real number t satisfies 


^-tH l+t {A\E\P AE ) = R s , 
at 


(185) 
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which implies t = s > 0 due to the definition of R s in (24) and the strictly decreasing nature of R s . As a result, 


P 


AE 


| (a, e) : P A | B (a|e) > c 0 e = exp (-n {sH 1+s (A\E\P ae ) - S-R s j) . (186) 


Plugging this into the left-hand-side of d 1 83[ ) yields the claim. The one-shot bound in ( |115[ ) with c = coe nR =+ nR > 
1 implies that 


e sC 1+s (f(A™)\E"\PX B ) 

> P AE {(a, e) : P^(a|e) > coe^ } c s 0 e~ n ^e snR 

+ 1 — (1 + s )Pae |( a ) e) : P2| E (a|e) > coe Ti ^ s | (187) 

= 1 + (c s 0 e~ snR °e snR - 1 - s) P n AE {(a,e) : P n A |J5 (a|e) > coe""^} . (188) 


Hence, taking the logarithm of d 1 88[ >, employing the lower bound log(l + b) > b — the large-deviations result 

3 “ s nRs+snR _ ^ _ S (_^ s _)_ Rf we obtain 


(184), and the fact that lim,, 


n lo S( c 0 


sCi+s{f (A n )\E n \P AE ) > exp [-ns(H 1+s (A\E\P AE ) - R)} . (189) 


Finally, we obtain the upper bound to ( |55] ) by taking another logarithm and normalizing by n. 
For the other case R < R s , we claim that 


max {tH 1+t (A\E\P AE ) - tR} = max {tHi +t {A\E\P AE ) - tR}. 

t>s t >0 


(190) 


This is because by the strict concavity of t, i-)- tH\ +t , the map s t-A R s is strictly decreasing. So for R < R s the 
maximum on the right-hand-side of ( |190[ ) is attained at some t > s. This is also reflected in Fig. [5] Thus, ( |112[ ) 
implies that 


e sC 1+s (f(A^)\E n \P2 B ) 

>Pi‘ £ {(a,e):PJ |E (a|e)> Co e-” B }cS 

+ (l - n E {(a, e) : P” |E (a|e) > coe-”'*})‘ + “ 
> 1 + W - 1 - >)Pae {(a,e) : PJ |B (a|e) > c 0 e~"- R } . 

Hence, 


(191) 

(192) 


Ci+s(f {A n )\E n \P AE ) 

= l\ og t e ^ 1+ s(f(A-)\E-\P2 B h 
s 

> \ log (l + (eg - 1 - s)Pl E {(a, e) : P% E { a|e) > c 0 e—*}) 
= Cf) ~ l s ~ S PAE {(a,e) : P^ E ( a|e) > coe"™*} . 


(193) 

(194) 

(195) 


By combining the asymptotic evaluation using Cramer’s theorem in ( |184[ ) and the equality in ( 190 ), we see that 
for R < R s , we also obtain the upper bound to ( |55j ). This completes the proof. 

b) Proof of the upper bound of & The proof of the upper bound to ( [57] ) is similar and we present the 
details here. Similarly to the above proof, choose c = c 0 e~ nRts+nR > 1 and the constant cf > 1 + s. Assume that 
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R > III , where III is defined in ([25]). Then the one-shot bound in (| 11 6[) implies that 


e I^C?+.(/(A")|£"|P2 B ) 


>^P£(e)(p^ |E „ =e {a:P” |E (, 


a|e) > coe nn ° j c^e 
l+s 


—nKl [ snKl gSnR 


+ (l - P A n\ E «= e {a : F% | E (a|e) > c 0 e ) 1 


—nRt \ „s —snRl snR 

Cq C C 


> E^< e )(^-I®—{« : PJ| E (a|e) > cue""*} 

e 

+ 1 — (1 + s)P A n\E*=e |a : P'f | S (a|e) > coe nR - j j 

= Pe{*) (l + (cfc- snk h snR - (1 + s))P An]E 


{a : P A \ E { 


a|e) > coe 


-nRl 1 \ i+ 


r 


E *SC) (l + +s) p A ^ | a : P „ B(a|e) > coe -„H;} ) 

e 

1 + C °e 8nR ‘ e ° n *~ (1 + £) ^Pg(e)P A „| E „ =e {a : P2 |E (a|e) > coe""*!} , 


where ( 199 ) follows from the fact that log[(l + ay] = t[a + 0(a 2 )] for a l 0. Hence, 

c; + ,(/(A”)|£”|Fi E ) 

= 1 + 8 ] og LTk C U.lf( A ")\E~X,\PS m xP x .) 


> 1 + S log (l + c s 0 e~ snRa e snR - (1 + s) 
s & V 1 + s 


E me)PA- |E— {« : ^| E (a|e) > coe-*} 1 


^ c s 0 e~ sn ^e snR - (1 + s) 


^P£(e)P A „ |E „ =e {a : P^ |B (a|e) > coe”^} , 


c s 0 e~ snWa e snR - (1 + s) 


P 


AE 


{(a,e) : P^ £ (a|e) > c 0 e nR '} , 


(196) 

(197) 

(198) 

(199) 

( 200 ) 


( 201 ) 

( 202 ) 

(203) 

(204) 


where ( |204| ) follows from log(l + a) = a + 0(a 2 ) as a J, 0 and the fact that the summation in ( |202[ ) vanishes as n 
grows. Combining ( |183[ ) and ( |204[ ) yields the upper bound to (57) for R > III . A similar calculation for the case 
R < Rl also yields the the same upper bound to ([57]). 

c) Proof of the upper bound of (56): We choose the constant c such that (1 — s) > c~ s . We apply Cramer’s 
Theorem [|48| to the sequence of random variables log P'^ E {A" \E n ). Then, 


1 


lirri - - l ogP AE { (a,e) : P% | S (a|e) > } = max {tH 1+t (A\E\P AE ) - tR} . 


nR 


t> 0 


n—>• oo Ti 

The one-shot bound in ( 1 \2\ implies that 

e -«C 1 _.(/(A»)|£^|PJ a ) 

^-PAp{( a i e ) : ^A|£;( a l e ) — j\/} C S + Pae\{^ e ) : -PA|p( a l e ) < 


(205) 


1 —s 


- P Ap{( a i e ) : ^A|E( a l e ) ^ S + (■*■ ~ jP Ap{( a i e ) : P A\E( a \ e ) P Jf}') 

< 1 - «1 - ») - c~‘)Pa E {( a,e) : P\ |E (a|e) > A}. 


1 — S 


(206) 

(207) 

(208) 
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Thus, 


Ci—s(f (^4 n ) | E n | Pae) 


= --log 
s 

> — log 
s 




> 


!-((!-»)- c-')Pi E {( a,e) : P} |B (a|e) > 2j} 
1((1 “ ») “ C"') P 5E{( a . e ) : Pl|E( a l e ) > Jj]< 


(209) 

( 210 ) 
( 211 ) 


where the final step uses the inequality log(l — t)< —t. Combining the limiting statement in ( 205 I and the bound 
in ( |211 ), we have the upper bound to ([56]). 

d) Proof of the upper bound of ( |58j ).- When e p e ( e ) P A n \ E » =e { a : T*2| B ( a | e ) > jf } is exponentially small, 

(TT3) implies that 


e -j^Cl_ff(A-)\E-\P2 E ) 

<J2 P E( e )( P A"\E*= e {a--PA\E( a \ e ) > 


+ -PA"|E-=e{ a : ^4|p( a l e ) < Jf } ^ 

E^(e)(c"^-|^{a:PS |B (a|e) > 


l-s\ 1^7 


+ (l - Pa" |E”=eja • Pl|E( a l e ) - jv/}) j 

< ^FB(e) (l - (1 - »- OfV|E-_.{a : PJ |B (a|e) > 


= £Pg(e) 1 


1 - 1 1 8 _ s C ~‘ EPSM P -«-IE-.{a : PJ |B (a|e) > £}, 


1 — s 


~ p A n |P"=e{ a : ^A|p( a |e) > ^} 


where (|215[) follows from the same reasoning as (|199[). Thus, 


Cf_,(/(A")|E"|PI e) = -^ log [• 


^fk C Ls(f(A n )\E"-\PZ B ) 


1 — s , 

>-log 


1 - 


1 — s — c 


1 — s 


Y, p 2(e)P A , IE ,..{a : PJ| B (a|e) > 2j} 


> 


1 — s 1 — s — c 


s 1 — s 

1 -S-C~ s 


E^(e)^|^= e {a:P^(a|e)>^} 


Y,PZ(e)PA^=e {a : ^| E ( a |e) > ^}. 


( 212 ) 


(213) 

(214) 

(215) 

(216) 


(217) 

(218) 

(219) 

( 220 ) 


Combining (|205 ) and (220), we have the upper bound to (581. 


C. Proof of Theorem [7] 
1) Direct Parts: 
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a) Proof of upper bounds for Case (A): First, we prove the upper bounds for Case (A) where the Renyi 
parameter a = 1 + s for s G (0,1]. Substituting e nH 1+s (A\E\p AE )+fnL j nto j n the c hain Q f inequalities in 
( |1 18 [ )—( p~T9| ) . we obtain, for the class of e-almost universal hash functions fx „, that 

Ci+s(fx„(A n )\E n X n \P2 E X P Xn ) < \ log {e s + e s ^ L ). (221) 

Set e to be a constant (not varying with n). Normalizing by Jn and taking the limsup as n —> oo yields the upper 
bound to ( |68| ). 

In an exactly analogous way, the upper bound to (691 can be shown by substituting e nH i +a ( A \ E \ p AE)+fnL j nto 
M n in the chain of inequalities in ( 176 1—( 178 ). 

Substituting e nH 1+s (A\E\p AB )+y/nL j nto j\f n j n q lc chain of inequalities in ( 173 )-( 175 ) with t = s, we obtain 

Ci +s (fx n (A n )\E n X n \P% E x PxJ < M^e~ snHl+ ^ A ^ PAE) = e S ^ L (222) 

which implies the upper bound to ( [70] ) after we take the logarithm, normalize both sides by Jn and take the lim sup 

as n —> oo. 

In an exactly analogous way, the upper bound to ([77]) can be shown by substituting e nH i+s( A \ E \ Pa )+£ nL into M n 
in the chain of inequalities in ( 1 20] )— ( 121 1 . This completes the proof for the direct part of Case (A) of Theorem [3] 

b) Proof of upper bound for Case (B): Case (B) follows from four distinct steps, detailed in each of the 
following paragraphs. 

In Step 1, we fix any function / : A n —»-{l,...,||/||}. We partition the space A" x £ n into pairs of sequences 
of the same joint type [14]. Let Qae denote a generic joint type on A x £. Let I/Aae ) he the uniform distribution 
over the type class Tq ae C A n x £ n . Let 




(223) 


be the distribution on 


a:/(a)=j 

x £ n when the hash function / is applied to the variable A n and denote 


ui 


’(e):=^C 


}{AAA 


{hj 


(224) 


Z— 1 


as its £"-marginal. Because the probability of pairs of sequences of the same joint type have the same P'[jt- 
probability, we can write 


(225) 


U 


(Qae) 


A n E n 


Pf(An)Mhe) = E n E (.T QAE )U^ En (i,e). 

QAE^'Pn(A.XS) 

By using ( |225| ), we have 

Ci(f(A n )\E n \P2 E ) = D{P KA ^ E n\\P^,f(AA x PeJ 

< E p ae(TqAD (ufA,E- x V l A ] 

QaeCPAAxS) 

= E P ae{T Qae )Ca ( f(AJ\E 

QAE^V n {AxS) 

< 

QAEeVn(AxS) 

where ( |227[ ) follows from the fact that relative entropy is convex, ( |228| ) follows from the definition of C\, and 
(229]) follows from the fact that s h->■ Cj + . s is monotonically non-decreasing. 

In Step 2, we regard / as a universal hash function fx„ ■ Thus, ( |229| ) implies that 

Cjf x JAJ\E n X n \Pl E x P X J = E x „ [CJfxM n )\E n \Pl E )] (230) 

< E P ab(T Qae ) E x „ [c 2 (. fx n (A n )\E 

QAE^Vn (Ax£) 


E P ae(T Qab )C 2 [f(AJ\E n U 


(Qae) 

A n E n 


(226) 

(227) 

(228) 
(229) 


pn jj 


(Qae) 

A n E n 


(231) 
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Let 7Q A|B (e) := {a : (a,e) e Tq ae } be the conditional type class of Q A \ E given e, also known as the QyUE-shell. 
By the method of types EB Ch. 2], we know that for e of type Qe, 

log|T QA|J5 (e)| = nH{A\E\Q AE ) + 0(logn). (232) 


By using the fact that e H A A \ E ) is the conditional collision probability (i.e., e H A A \ E ) = E e P E (e)P A A'\E=e{( a , a ') '■ 
a = a'} where A, A' are conditionally independent and identically distributed given E), 


e -H 2 (A\E\u^A B )) = ^ u^ AB \e) Y 




a6T Q J!E ( e ) 

1 

' \TQa\e ( e )l 


l'We )| 2 


= e -nH(A\E\Q AE )+0(logn) 


(233) 

(234) 

(235) 


Furthermore, by a Taylor expansion of H(A\E\Q ae ) around Pae as in the rate redundancy lemma [511, [61], we 
have 


H{A\E\Qae) = H(A\E\P ae ) + Y^ AE ^ a ' e ) " PA E (a,e))h AlE (a\e) + 0(\\Q A e ~ Pae\\ 2 ) (236) 

a,e 

where the conditional entropy density h A \ E (a\e) is defined as 

hA\E(a\e) := log ——(237) 

PA\ E {a\e) 

and || Q - P|| = | Q(z) — P(z) | is the variational distance between Q and P. For brevity, we denote the 


(238) 


n-scaled version of the second term in ( |236[ ) as 

b n {QAE ) := yfn e) ~ p AE{a,e))h A \ E (a,e)^ . 

If Qae is a random type formed from n independent copies of Pae, 

b n (QAE ) = ^ Y hA \ E (Ai\Ei) - H(A\E\P ae A N (0, V(A\E\P ae )) 

by the central limit theorem. That is, b n {Q AE ) converges in distribution to the Gaussian M (0, V(A\E\ P AE ))- 
In Step 3, we first fix 5 > 0. Applying the universal property of the universal hash function fx„ to the collision 
relative entropy (see ( |118[ )-( fTT9| ) with e = s = 1), and combining the above notations and bounds, we obtain for 
all e G Tq e and all n large enough (depending on 5) that 


(239) 




C 2 (fxAA n )\ 


E r ‘ 


U 


(Qae) 


A n E " 


= Ev 


logM - H 2 (f Xn (A n )\ 


E r - 


U 


(Qae) 


A 71 E 71 


< log (l + M n e~ nHiAlE \QAE)+0(iogn) 

< log (l + exp [\/n(L - b n (Q AE ) + o{b n (Q ae)) + 0(log n )]) 

< 


(240) 

(241) 

(242) 

(243) 


Vn(L - b n {Q AE ) + o{b n (Q AE ))) + O(logn) b n {Q A E ) < L + 6 
Q-S^/n/2 b n {Q AE )>L + 5 ’ 

where ( 240 ) follows from the definition of C *2 and ( 241 1 uses the bound in ( |235| ). Also note that we used the fact 
that ||/|| = M n = Q nE (A\E\p AE )+y/nL in ( [242| ). 

Finally in Step 4, by plugging (|243[) back into (|23 1 [). we obtain that for all n large enough (depending on 5), 


C 1 (fx n (A n )\E n X n \P%E x PxJ 

< Y PMT QAE ){L-(l-5)b n (QAE))+o( l -^Y (244) 

Q AE £PAAx£):b n (Q AE )<L+6 ' V ' 
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Let V := V{A\E\P ae ). By the central limit-type convergence in (|239|), we obtain 


limsup ~^=Ci(fx„{A n )\E n X n \P2E X PxJ < 


r L +s L _ (! _ g ) b _ 

-oo \/27rF 


b 2 /(2V) dk 


(245) 


n—)-oo V Tl 

By a change of variables to x := b/s/V and taking 6 j. 0, we immediately obtain the direct part (upper bound) of 
Case (B) in 


c) Proof of upper bounds for Case (C): For Case (C), the upper bound to ( |73j ) can be obtained by specializing 
the one-shot bound in < |107| ) with e = 1, M n = e nH (A\E\p AE )+fnL an( j c _ e n _ With these choices, we have 

a—sCi- a (fx n (A n )\E n X n \P AE x Px n ) 


_ —n 1 / 4 


^ P'ae{ ( a > e ) : ^4|e( a l e ) ^ 


^nH{A\E\PAE)-\-\/nL J \ 1 -|- 6 


1 


- a — n 1 / 4 


(246) 


= P 


f -j N -j ^ 

| (a,e) : -= ]T [ - log P^KN) - H(A\E\P ae )\ > L + -= i (247) 

r v v j 


AE 

The probability is an information spectrum [471 term with n independent and identically distributed random variables 
and since P Ai Ei = Pae for each 1 < i < n, 

E p AiBi [ ~ logP AlE (Ai\Ei)] = H(A\E\P ae ), (248) 

Varp A . E . [ - logP A \ E (Ai\Ei)] = V{A\E\P ae ). (249) 

So by the central limit theorem, the right-hand-side of (|247[) converges uniformly as follows: 


f 1 . n . 1 ) 

}^ Pae \ ( a ’ e ) : i ~ lo S PA\Ei a i\ e i) ~ H(A\E\P AE )\ > L + —= 

^ v i =1 v ^ 


= $ - 


L 


--)■ (250) 


y/V(A\E\PAE)' 

Plugging ([250 1 into ( |247[ ), taking the logarithm, and normalizing by — s yields the upper bound to ( f73[ ). 

In a similar way, the upper bound to ( f74] ) can be obtained by specializing the one-shot bound in ( 108 ) with 
e = 1, M n = e nH ( A \ E \P ae)+VuL an( j c _ calculation for the specialization is similar to the converse 


part which is detailed in full in ( |280[ )—( |283[ ) in the next section. This completes the proof for the direct part of Case 
(C) of Theorem [3] 

2) Converse Parts: 

a) Proof of lower bounds for Case (A): We now prove the lower bounds for Case (A). The first two bounds 
can be shown using the data processing inequalities in (|26l)-([28|). In particular, the lower bound to ( [68] ) can be 
evaluated as follows: 


C 1+s (f(A n )\E n \P2 E ) = nH 1+s (A\E\P AE ) + V^L - H\ +S {f (A n )\E n \P AE ) 

> nH 1+s (A\E\P AE ) + sfnL - H 1+s (A n \E n \Pf E ) 

= nH 1+s (A\E\P AE ) + y/nL - nHi +s (A\E\P AE ) 

= y/nL, 

where ( 252 ) follows from ( [27] ). The lower bound to ( |69| ) follows completely analogously using (28). 

The lower bound to (f70[) can be shown by first relaxing (|114[) as follows: 


(251) 

(252) 

(253) 

(254) 


e sC 1 + s (P A )\ E \PA E ) 

> J2 PE(e)P A]E (a\e) 1+s M s + Y,P E (e)P A \ E=e {(a,e):P AlE (a\^ 

(a,e):P A \E(a\e)>-^ e 

C l!+ s 


< 


c d+s 
M J 


> ^2 PE(e)P A i E=e {(a, e) : P A \ E {a\e) < 


M J 


> 


-1\ 


P AE \{a,e) : P A \ E {a\e) . mJ 
1 - P AE \(a,e) : P A \ E {a\e) > 

(1 + s)P AE { (a, e) : P A \ E (a\e) 


1 l+ s 


> 1 - 


Mil 

>-} 

“ Ml 


(255) 

(256) 

(257) 

(258) 


(259) 
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where ( |257| ) uses Jensen’s inequality (for the convex function 1 H > t 1+s ) and ( |259[ ) uses the inequality (1 — x) 1+s > 
1 — (1 + s)x (also due to the convexity of t i-> t 1+s ). Hence we have for the n-shot setting 

(260) 

Applying the modified Gartner-Ellis theorem derived in Hayashi-Tan [ 621 Appendix A] to the sequence of random 
variables — log P^ E {A n \E n ) with M n = Q nH i+a{A\E\ p AB )+fnL an( j c = we have 


sC 1+ ,(f(A n )\E"\P1 e) > log ( 1 - (1 + s)Pij;{(a,e) : PJ |E (a|e) > g-} ) . 


loeP A ^ e): a if) 


= — sL. 


(261) 


The modification here is due to the different normalization of s/n as opposed to the normalization by n in the 
usual Gartner-Ellis theorem in | [48| . Also see Remark (a) to Theorem 2.3.6 in 1481. Combining ( |261[ ) with ( |260[ ) 
yields the lower bound to ([70]). The lower bound to ( |7T| ) can be proved in a completely analogous way by relaxing 
the one-shot bound in ( 1 16| ). 

b) Proof of lower bound for Case (B): For the converse part of Case (B), we use Theorem 8 of [ 161, which 
analyzes the second-order asymptotics of intrinsic randomness [47 Ch. 2] [631. Define the second-order coding 
rate at length n as 

1 


L n := —= (log M n - nH(A\E\P AE )) 


(e) 

and the distribution function Fn which is dependent on e as 


Fi e \x) := P A n lE n =e l a : ~ logP^, s „ =e (a) < H{A\E\P ae ) + -j= \ . 


(262) 


(263) 


Now, from the proof of Theorem 8 of [161 (second column page 4634), we deduce that for each c 7 £' 

H(f(A n )\P A n lEn=e ) 

f L n 

< / adF^\a) + nH(A\E\P AE ) 


+ P, 


A-|E"=e 


Now note that Fn ) (x) depends only on e through its type. Our next step is to take the expectation of ( |264| ) over 

1 ‘ (265) 


-Aa-.P 


A n\ E n =e {<X) ^ 


^ \ V " Ln ~ '° gPA ~\ E — \ B : F ^|E-="< a ) - i/) ' 


(264) 


e with distribution Pg. Let 


(266) 


5(e) := P4-|E"=e |a : P M |£>=e( a ) < jj- 
Since t (->• — t logf is concave, by Jensen’s inequality, we have 

E Pe „ [g(E n )( 7 - log g{E n ))} < E Pe „ hg(E n )} - E Pe „ [g(E n )j logE PE „ [g(E n )}. 

Now define the averaged distribution function as 

F n (x) := £pg(e)4 e >(x) = P n AE {(a,e) : - 1 logPg |£ (a|e) < H{A\E\P ae ) + *= 
e l n v n 

Let 7 := \[nL n . From ( |266[ ) and the definition of F n (x), 

H(f(A n )\F n \P2 E ) 

< y/n [ adF n (a) + nH(A\E\P AE ) 

J — OO 

+ Pae {( a i e ) : ^A|£;( a l e ) — YF~\ (s/nLn — log P AE |(a,e) : Pg| P (a|e) < —— 


(267) 


(268) 
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Thus, by invoking the definition of L n in (|262|) and F n in (|267[), we obtain the inequality 


-= (H(f(A n )\E n \P2 E ) - nH(A\E\P ae )) 


n 


< 


' — OO 

pL n 


(a e) < 


adF n (a) + P AE <j (a, e) : P'f 1 ^(—, — y ^ 

log(l - F n (L n )) 



!og P AE {( a > e ) : P A\ E ( a \ e ) ^ i} 


n 


' —OO 


adF n (a) + (1 - F n {L n )) - 

By the central limit theorem 

F n {x) -> F(x) = 


n 


—00 V 2 ttV 


i y2 d 2V )d y, VxG 


Taking the limsup of ( |27Q[ ), and using the central limit result in we obtain 

limsup (H(f(A n )\E n \P2 E ) - nH(A\E\P AE )) < T adF(a) + L{ 1 - F(L)). 

n—too V n 7-oo 

Since, we have the simple relation 

Ci(f(A n )\E n \P% E ) = D{P f{An)tEn \\P^ tf{M) x P E „) 

= —H(f(A n )\E n \P AE ) + logM n 
= — H (f (A n )\E n \P AE ) + nH(A\E\P AE ) + V^L, 

we immediately obtain the desired lower bound for the second-order asymptotics of C±: 


lirninf —=Ci(f (A n )\E n \P AE ) > 

n—foo x n 


(L — a) d F(a) = 


rL '^ L -fx ^n^ 

— 00 v 27 T 


(269) 

(270) 

(271) 

(272) 

(273) 

(274) 

(275) 

(276) 


c) Proof of lower bounds for Case (C): For Case (C), the first part of the maximum in the lower bound in 
& namely Ti(s,L) in ( |64| ), follows from ( | 1 1 7[ ) and the second part of the maximum, namely ITf-s, L) in ( [65] ), 
follows from ( | 11 2[ ) with the common choice of c = e nl/J . In particular, specializing the bound in one-shot bound 
in (112) with this choice of c, we obtain 

e sCis(f(AA\EAP2 E ) < ( e " 1/4 )- s + P"J (a, e) : Pf E (a | e ) < 


l.i/4 


inH(A\E\P AE )+yfnL 


1 —s 


(277) 


where we trivially upper bounded the first probability in the one-shot bound by 1. The first term in ( |277[ ) goes to 
zero (since s > 0) while the second term is an information spectrum term that asymptotically behaves as 


1.1/4 


lim Paf^ (a, e) : -P 4 ip(a|e) < — — —— —-—■=— 
1 —^00 AE \ y ’ ’ A \ EK 1 J ~ e nH(A\E\P AE )+yfnL 


= $ - 


L 


V V(A\E\P ae )) 


(278) 


by the central limit theorem and the statistics computed in ( 248 >—( 249 1. Hence, taking the logarithm in ( |277| ), and 
normalizing by — s, we obtain the second term in the maximum in the lower bound in ( [73] ), namely T 2 (s,L). In 
exactly the same way, specializing the bound in ( 117 ), we obtain 

e -»C 1 _.(/(A»)|B»|J^ a ) < ( e nl/4 ) _s + 2^s^PIe{ (a, e) : P^(a|e) < 


7. 1 / 4 


,nH(A\E\P AE )+yfnL 


(279) 


Applying the central limit theorem to the probability in the second term recovers T 1 (s, L) in the lower bound 
in ( [7^] ). 

The method to obtain the two terms in the maximum in the lower bound in ( |74| ) is more complicated than 
that for ([73]) because we need to condition on various sequences e e £ n . In particular, to obtain the lower bound 


T'i(s, L) in (66), we evaluate ( 110 ) with c = e n . We obtain 

e ~T^Cts(f( An )\E n \P2 B ) 


< 2 I3 J 


(e”‘"r* + £ Pg(e)(2* S *(1 - S )P A .| E .«{a : PJ |E (a|e) < ^-}) 


(280) 
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As usual, the first term goes to zero. To compute the probability in the second term, let us denote the type (empirical 
distribution) [ 141 of e by Q e £ V n (£) for the moment. Then we have 

L + H(A\E\P ae ) - H(A\E\P AE \\Q e ) 


P 


A n\ E n=e 


{ a : p a\e( 


a e < 


M r 


■}-*(' 


y/V 2 (A\E\P AE \\Q e 


<0 [-=) (281) 


by the Berry-Esseen theorem [64 Sec. XVI.7], where the conditional entropy given another distribution Q e , denoted 
as H(A\E\P AE \\Q e ), was defined in < [T2| ), and conditional varentropy given another distribution Q e is defined as 

1 12 


V 2 (A\E\P AE \\Q e ) := £ Q e (e) £ P A]E (a\e) 


log 


p a\e{ 


a e, 


- H(A\P A]E=e 


(282) 


Note that V 2 (A\E\P A e\\P E ) = V 2 {A\E\P ae ) defined in (62). In ( |28 1 [ ), the remainder term 0{^=) is uniform in 
L and Q e . We now plug this into ( |280[ ) and notice that we are then averaging over all types Q e (where E n has 
distribution P E ). Now, employing a weak (expectation) form of the Berry-Esseen theorem [65 Thm. 2.2.14] with 
x = H(A\E\P ae ) - H(A\E\P AE \\Q e ) yields 

e -M-s C lsU{AA\E-\Pi E ) 


< 2~‘ 


o(n (1 — s)^ 


f 


$ _ 


L -\~ x 




+ o 


^v 2 {A\E\p AE ) J \y/fU) ^vmieipae) 


_L_ e -x 2 /(2V 1 (A\E\P AE )) 


: dx 


+ o(^ 

'n 


(283) 


Now we take the logarithm, divide both sides by — and take the limit as n oc. This yields the lower 
bound T'i(s,L) in ( |66| ). Note that here unlike in the steps leading to ( 100 1 , we cannot add V\ and V 2 due the 
exponentiation of the first term by y- 1 — in the integral. 

Using similar techniques, we can obtain the lower bound (s, L) defined in ( |67| ) from ( |1 13[ ). In particular, 
evaluate ( 113 ) with the same choice of c. Here, in fact, no averaging over E n is needed because the first term 
in ( 1 13[ ) vanishes by our choice of c = e” 14 . Thus, we obtain the lower bound in ([74]). 

This completes the proof of the converse parts of Theorem [3] 


VIII. Conclusion 

A. Summary 

We have derived the fundamental limits of the asymptotic behavior of the equivocation when a hash function 
/ is applied to the source (Theorem [I]). We have also showed that optimal key generation rates change when 
we use alternative Renyi information measures (Corollary [T]). Under these Renyi quantities, we have evaluated the 
corresponding exponential rates of decay of the security measures (Theorem [2]) as well as their second-order coding 
rates (Theorems [3] and [4]). The Renyi information measures generalize the ubiquitous Shannon information measures 
and may be useful in many settings as described in the Introduction. To establish our asymptotic theorems, we have 
introduced new families of non-asymptotic achievability and converse bounds on the Renyi information measures 
and their Gallager counterparts and used various probabilistic limit theorems (such as large deviation theorems and 
the central limit theorem) to evaluate these bounds when the number of realizations of the joint source tends to 
infinity. 


B. Future Research Directions 

In the future, we plan to explore various extensions to the results contained herein. 

1) We would like to study security problems such as the remaining or residual uncertainty of a source A n when 
another party observes a compressed version f(A n ) £ M. := {1,..., M n } and another correlated source E". 
Namely, we aim to study the asymptotic behavior of the conditional Renyi entropy Ffi +s (A n \f(A n ), E n \P^ E ) 
and its Gallager counterpart H\ +s (A n \f{A n ),E n \P^ E ). 

2) Another set of related problems involve the analyses of the asymptotic behavior of PLi +s {f {A n )\E n \P^ E ) 
and H\ +S {f iyA n )\E n \P^ E ). These represent the uncertainties of an eavesdropper with regard to the message 
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3) 


4) 


index f(A n ) <G A4. The eavesdropper, however, is equipped with correlated observations E n . We anticipate 
that some of the techniques developed in the current paper may be useful to perform various calculations. 
We focused primarily on analyzing C\+ s and Cj, s for s £ [—1,1]. It may be of interest to study the various 
asymptotic behaviors of C\ +s and C\ +s for general s£l since for example, H mm = lirm^oo H\ +s and 
fT m in lj 6j-|(9 | is a fundamental quantity in cryptography and information-theoretic security as mentioned in 
Section 


I-A 


Indeed, q~ h ^A a \ e \ p ae) j s the ^ est (highest) probability of successfully guessing A given E. As 
remarked after Theorems [T] and [3] we already have the converse parts for all s > 0 for the results in ( |47| ), 
((48]), ( |68| ) and ([69]). They follow immediately from various information processing inequalities. It would be 
ideal, though challenging, to complete the story. 

Lastly, we aim to apply the results and techniques derived herein to information-theoretic security problems 
such as the wiretap channel |T2j and secret key agreement |[T9|] as was done by various researchers in (17|, 

GD, ©, ED- 


Appendix A 
Proof of Lemma Q] 


A. Proof of ( |102[ ) 

Proof: The derivation here is similar to that in (17) , |22| for universal hash functions. Throughout, for any 
function / : A —> M., we let 


/ 1 (i) := {a € A : /(a) = i}, VieAL 
Now, for any a, due to the e-almost universal property of f x , we have 

IE A' ^ P A\ E ( a '\ e ) < P A\E( a \ e ) + JJ ^ P A\E( a '\ 

a'£fx 1 (.fx(.a)) a '+ a 

< p A\ E {a\e) + jj- 

Starting from the definition of the conditional Renyi divergence, we have 


^-sH 1+B (f x (A)\EX\P AE xP x ) 


M 


E x Y Pe ^Y\ ^2 P A\E(a\e) 
1=1 


1 +s 


E^wE^vt 


a e 


t: p A\E{ a> \ e ) 

<^2 p E(e)^2 p A\ E (a\e)(E x ^ p A \ E (a'\e) 


<E p Ae)E P A\ E { a \ e ) f P A\ E { a \ e ) + JJ 
e a ' 

< Y Pe ( e ) Y P A\E(a\e) ( p A \E(a\e) s + 

a ' 

+Y Pe ^Y Pa \e( 


e 

M 

s 


[a e 


,1+s 


— __|_ p -sH 1+s (A\E\P ae ) 


(284) 

(285) 

(286) 


(287) 

(288) 

(289) 

(290) 

(291) 

(292) 

(293) 


where ( 289 1, we used the concavity of t i-a t s for s 6 [0,1], in ( |290| ) we used the fact that f x is a e-almost 
universal hash function, and in ( 291 1 we used the inequality (JA a,i) s < JA of for s £ [0,1 A3 Problem 4.15(f)]. 

We remark that the sequence of steps in ( |287[ ) to ( |293[ ) is inspired by the work of Hayashi 1171 who derived a 
similar result but for Shannon-type quantities instead of Renyi-type quantities as we do here. 
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By ( [34] ). we have 

C 1+s (f x (A)\EX\P AE x P X ) = log M - H 1+s (f x (A)\EX\P AE x P x ). 
Uniting ( |293[ ) and ( |294[ ) proves d 1 02| ) as desired. 

B. Proof of ( |103[ ) 

Proof: Along exactly the same lines, we also have 

e - xfjpH?+. (fx (A)\EX\P AB x P x ) 


M 


l+s\ i+ 


E.y E P ^>(E( E 

i=1 aef-fi) 


= E. y p A\ E {a\e) ( P a\e(o,'\ 

e V a a,'efxff x (a)) 

<J2 P E(e)(j2 P A \E^)(^X P A\ E { a 'V 

1 

£ \ s\ 1 + 


<Ew E p A\ E (a\e) y p A\ E {a\e) + M 

e 'a 

<J2 Pe ^(J1 p A\E{<Ae)(PA\E{a\e) s H 

e 'a 

= J2 PE ^((if) +Y, p A E ^ a i e ) 

e ' a 

£ E p ^)((s)* + (E P A|E(«|e) 1+ -'- + 

' a 

° +Y / p E (e)(Y / p A\ E ( 


e \ s \ \ '+» 

M, 


\1 + S 


e 

M 


gl + o 


a e 


,1+S \! + 


+ e -i^ H t+M\ E \ Pab) 


4 4 __ 

Combining this with the relation between /7 |+s and CJ +S in (|36J), we obtain (jJ_03J). 


C. Proof of ( ] I04[ ) 

Proof: For any a, we have 


^2 p A\E(a'\e) 

a '^fxffx{a)) 

< p A \ E {a\e) + P A\ E ( a '\ e ) 


M 

a'^a 

< P A \ E {a\e) + ~~ 

< 2max|p4| E (a|e), j. 

First we observe that when -P 4 |s(a|e) E jj, we have 

Ex ^2 P A\ E ( a '\ e ) < -P4|E( a l e ) + 
“'G/* 1 (/*:(“)) 


_f_ < c + e 
M ~ M 


(294) 


(295) 

(296) 

(297) 

(298) 

(299) 

(300) 

(301) 

(302) 

(303) 


(304) 

(305) 

(306) 


( 307 ) 
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Now we have 


e -sCi_ a {fx ( A)\EX\P ae x P x ) 

1 r sH 1 _ a (f x (A)\EX\P AE xP x ) 

M s 

- Y P A\E(a'\ 

e a a'efx^fxH) 

> ^PAE(a,e)(2max{MP A l E (ale),e}) 


> 


2 s Y p AE(a,e)min{P AlE (a\e) S M s ,e s } 


= 2~ s Y PAE(a,e)P A]E (a\e)- s M~ s 

a,e:PA|£;(a|e)>eM _1 

+ 2 s Y^ PAE{0",e)e s , 

a,e-.P A f E (a\e)<eM~ 1 


(308) 

(309) 

(310) 

(311) 


(312) 


where in (3091 we used the convexity of x *->• x s where s G [0,1] and x > 0 and in (310). we used (306 >, Thus, 


we obtain (104). 


D. Proof of ( |105| ) 


Proof: Using (306) and the convexity of a h->■ a 1 we have 


3 ifl (A)\EX\P ae xP x ) 
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^xY P e ^\ Y ( E P A\E(a\e 
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e ' a ' 

> Y PE ^(^ PA \ E ^ e ^{ 2max { PA \ E ^ e ^ jf}) 

e ^ a 
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e ' a 

= 2~^Y P E(e)f Y PAwiale) 1 -’+ 


— S \ 1-s 
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(314) 

(315) 
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(317) 
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2-T^P E (e)( E ^VHe) 1 " 

e a:P A \B(a\e)>^ 

-PA|s(a|e' 

C e a:PA|E(a|e)<i 


(322) 


Thus we obtain (105 I. 


Appendix B 
Proof of Lemma [2] 


A. Proof of (1061 


1 i 1 

Proof: Since (1 + x) *+ a < 1 + and i o i»* is concave for s € [0,1], we have 


Mlf; H *+ a (/x {A) \ EX ]PaeX Px ) 
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= M^ExE^( e )(E( E P A\e(' 
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M^Ex^Pe^I E P a|e( 
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11 + S 


E Ps(e) (l + M s E P A \E{a\e) l+s 


< E P ^ e ) ( 1 + Y — s MS E P A\ E (a\e) 
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= 1 + rb M * S p ^ e ) E E^( a i e ) 

e a 

= ]_ _1_ M s e~ s Hi+b(A\e\p ab )' 

1 + s 


1+s 


(323) 

(324) 

(325) 

(326) 

(327) 

(328) 

(329) 

(330) 

(331) 

(332) 


Using (294 1 , we obtain (106 1 . 











Appendix C 
Proof of Lemma [3] 


A. Proof of ( 107[ i 


Proof: Using (307), we have 


Q—sCi- e (fx(A)\EX\P AB xP x ) 

= 1 c sH 1 . s (.f x (A)\EX\P AE xP x ) 


M s 

1 

Jf s 


M 


me 


1-8 


^ E xJ2 p E( e )J2( y Pa \ e ( c 

e i=1 aefx'ii) 

—Ex y pe ( c ) y PUisHe) ( y p/tya'i 


AP 


a'e/x^/xCa)) 


- P A|I?(o'| 

a'efxffxia)) 

p A\E(a\e)(E x y P A\E(a'\ 


a:P A \ E {a\e)< 

~^J 2 Pe ^ y Pa \ e ( 

e a:P A | B (a|e)< + 

= P AE {(a,e) : P A | E (a|e) <jj}( 


a e 


“'e/x'f/xW) 

C + 6N -s 


M 


1 \* 


c + e 


We obtain (107) as desired. 


B. Proof of ( ] 108| ) 


Proof: Using ( |307[ ), we have 

e T+; H \_. (f x (A) | EX | P AB x P x ) 
M 


^■x^2 p E(e)( y ( y PA|E(«|e 
V * =1 ae/j^O 


1 —S \ 1 - 


—S\ 1-s 


— S \ 1-s 


= E x ^P E (e)( ^P A , £ (a|e)( y P M a '\ 

V a «'£/i 1 (/x(a)) 

> 2 P E (e) f 2 P A | E (a|e) (e* y P A|e(«'N 
V a a,'efxffx(a)) 


> y, p E(e)( y, P4| E (a|e)^Ex y P^4|^(a / | 

a:PA|£;(a|e)<-^r a '€fx 1 (f X ( a )) 


— s \ 1 -s 


sEw E fW 

e a:P A | B (a|e)< 

'C + 6 


a e 


c + e\- 


M 


M 


1 _, yPp(e)( y PA|p(a|e; 

a:P A | B (a|e)< + 
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(333) 

(334) 

(335) 

(336) 

(337) 

(338) 

(339) 


(340) 

(341) 

(342) 

(343) 

(344) 

(345) 


By combining with (36), we obtain (108 1 . 
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A. Proof of ( |109[ ) 

Proof: Define the functions 


Then, we can show that 


Appendix D 
Proof of Lemma [4] 


gi(x,y) \=x + y-2x 1 s y s 
92 {x,y ) := x - 2x l ~ s y s . 

i _L__x 

mmgi(x,y) = x(l — 2 ‘-s 1 - (1 — s)), 

y 


which is attained when y = x{2s) 1 -°. We also define 

h e ,i(m) := Y p A\E(a\e ) 

he,2(m) := Y p A\E{a\e)- 

a S / _1 ( m )'-P A \E(a\e)>ff 

Hence, 

p f(A)\E{m\e) 1 ~ s = (h e ,i{m) + /i e ,2(m)) 1_s < /i e ,i(m) 1_s + he^m) 1 ^ 8 , 

which implies from the definitions of g\ and g 2 that 

9i[Pf(A)\E(m\e), > g2(h e ,2(m), + gi(h e ,i(m), . 

Also, we have 

he^m) 1 - 8 < Y PA\E( a \ e ) l s - 
n£/ _1 (m) 


Thus, 


1 _ 2e -^i-s(f(A)\E\P AE ) 

= Y Pe ^ ( Pf(A)\ E (m\e ), 


(346) 

(347) 


(348) 

(349) 

(350) 

(351) 

(352) 

(353) 


(354) 


' m 

V 2 W.T) 

m 

>) 


(355) 


2 (m) - 2 h £: 2 

(m) l - s M s ) + YKi^ 

7l)(l - 2^ 

sP-’il-s))^ 

(356) 

Y Pe ( g ) 

e 

Y heflim) 1 

171 

~ S M S - 2^s^(l - s 

)Y h e,l(m 

in 

) 

(357) 

Y Pe ^ 

E 

PA\ E ( a \ e ) 1S M S - 2^ 

*s^(l - s) 

1 Y P A\E(a\e) 

(358) 

e 

a:P A \ E (a\e)>^ 



a:P A \ E (a\e)< 


Y Pe ( e ) 

E 

P A \ E (a\e) l - s M s 




e 

a:P A \ E (a\e)>f i 





T = 7 S I = 7 (1 

- s)Pae|(ci, 

e) : PA|p(a|e) < Jf}, 



(359) 


where (355), ( 3561 , and ( 3581 follow from ( |352| ), ( |348[ ), and ( 353| ) respectively. Hence, we obtain ( 109 1. 
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B. Proof of (jTTO]) 

We first state a useful and easy lemma: 

Lemma 7. Let x, y > 0 and t > 1. Then we have 

(x + y) t <2 t ~ 1 (x t + y t ). 

Proof: It is clear that a a 1 is convex for a > 0. Thus, 

(* + »)* = 2‘(| + |)*< 2 ‘(^ + |)= 2 ‘-V + ! /‘), 

which proves the claim. 

Proof of d 1 10[ ): Using the previously proved bound in ( |109[ ) with \£\ = 1, we obtain 

e -sCl_ff(A)\p A \ E=c ) < c -s ^ P A ^ E (a\e) 1 ~ s M~ s 

a:P A[E (a\e)>^ 

+ 2^S^(l - 


(1 - s)P AE [{a,e) : P A \ E (a\e) <jj}- 


Taking average over P E and using the bound in ( |362| ), we have 

p ~T^CLAf(A)\E\P AE ) 


E p ®( 


e e 




< 


^2 p E(e) 


c~ s J2 p A\ E ( a \ e ) 1 ~ s M~ 

a:P A]E (a\e)>f= 


+ 2i- s i-(l - s)P AE {(a,e ) : T^pOle) < -^} 
<2^^P £ (e)[(c- s ]T P A \ E HeY- s M- s )~ s 

e a:P A | B (a|e)>-^ 

+ (2^s^(l - s)P AE {(a,e) : P A \ E {a\e) < ) 1_ 


(360) 


(361) 


(362) 


(363) 


(364) 


(365) 


where in the last step, we applied Lemma jT] with t = { 1 - > 1 to the term in parentheses in ( |364[ ). Thus we 
obtain d 1 10 b. ■ 


Appendix E 
Proof of Lemma [5] 

The inequalities in Lemma [5] can be shown by the information processing inequality for Renyi divergence in ( fTT| ). 


(366) 


A. Proofs of ( ] 11 1| ) and (112 1 

Proof: For every e G £, define the function f e : A -A At to be 

fe ■= argmin D\- s {P^ A ^ E=e \\P m hc,M )• 

We start with a claim that will be proved at the end of this subsection. 

Lemma 8. For every a such that P A \ E = e ( a ) > jw, we have \ ff 1 (f e (a))\ = 1. 

Now, we partition the set M into two subsets as follows: M i := f e ({a : P A \ E=e {a) > jf}) and M 2 '■= Mf. 
See Fig. [9j Next, we define the map g : M —> Mi U {0} as 


g{i) ■■= 


i if i € Mi 
0 if i E M 2 


(367) 
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{a : P A (a) > ^} {a : P A {a) < ■§} 



Fig. 9. Illustration of the steps in ( j368[ t to {372) where the dependences on e G £ are suppressed. 


Note that the map g depends on e £ £ (through Ad [ and Ad 2 ) but we suppress this dependence for brevity. 
Let PfjA)\E=e 0 S' -1 be the “output distribution” taking values on Adi U {0} induced by the “input distribu¬ 
tion” PfjA)\E=e and the “deterministic channel” g, i.e., for every i £ Adi U {0}, (Pf e (A)\E=e 0 5 f_1 )(*) := 
Pf B {A)\E=e{g~ l {i)) = P M A )\E=eU)- We also use the notation P miX)M 0 S' -1 a nalogously. Due to 

the information processing inequality for the Renyi divergence in ( |TT| ). we obtain 


0 — s (-^V e (A) | E — e || Pmix,A'd ) 

< 0-S-Dl- s (P/ e (A)|£; = eO^- 1 ||P m i Xl AdOC/- 1 ) 

iejojuAti 

= {( P MA)\E=e 0 g^ 1 )^)) 1 -* {(. P mix,M ° 5 _1 )(0)) S + E {(Pf.(A)\E=e ° 9 _1 )W) ^ (jj) 

i€M 1 

< P A \E=e{a ■ P A \ E {a\e) < ^} + E {(Pf.(A)\E=e 0 9~Y (*)) 1_a (jf) 

*eA4i 

= -P 4 |£; =e |a : -P 4 |£:(a|e) < —| + E P 4 | B (a|e) 1 

a:PA|B=e(a)>^ 


(368) 

(369) 

(370) 

(371) 

(372) 


where in (369) we used the definition of D\_ s , in (370 1 we split the resulting sum into {0} and Adi, in {371) we 
upper bounded (P m ix,A4 0 g _1 )(0) by 1 and we noted that all the symbols i £ M 2 = f e ({a : P A \E=e(a ) < jj}) 
are merged into the symbol 0 under g and finally in {372) , we used the fact that g and f e are one-to-one restricted 
to Adi and {a : P A \E=e( a ) ~ m } respectively (Lemma|8jand the fact that c > 1). See Fig. [9| for an illustration of 
these steps. Taking the average of {372) over Pe(P), we obtain {111) . 

Furthermore, we have 


E p A \ E { a \e) l - s M- s 

a-.P A \E=e(a)>^ 

< E p A\E(a\e)^y S M~ s (373) 

a-P A iE=c( a )>i^ 

= PA\E=e{a ■ PA\E=e(a ) > (374) 


Substituting {374) into the second term in {372) and then taking the average over Pe(c), we obtain {112) . ■ 

It remains to prove Lemma [ 8 ] 

Proof of Lemma [s| It suffices to consider the case \£\ = 1. Dropping the dependences on e, we denote f e as 
/ and P A \ E =e as Pa in the sequel. 
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We proceed by contradiction. The essential idea is that an optimal / (given by ( |366[ )) must induce a distribution 
Pf(A) on -Ad that is “as close to uniform as possible” since we are minimizing D\- s {P^a) ||-Pnix,.M)- 

Formally, assume, to the contrary, that |/ _ 1 (/(ai))| > 2 for some a\ G A with Pa{cl{) > h- Because 
l/ _ 1 (/(°i))l > 2, there exists a 2 7 ^ a\ such that f(a\) = f(a 2 ) = j for some j G M. Because Pa{ci 2 ) > 0, we 
have Ylaef- 1 ^) p A(. a ) > If - This in turn implies that there exists i G M. such that ^aef-qt) -Pa( a) < jj- Recall 
that / was designed to minimize 

Dl-s(Pf(A)\\PmhL,M) = -~ l0 g E ( E P A ( a )) (jff) > ( 375 ) 

or equivalently, to maximize JA ( zLae/-' (1) -P .4 (a))' *■ Now we create a new hash function 

/(«) := { / '“ ) “ ^ ■ (376) 

[ i a — a 2 

Let u,v,u',v' be any four non-negative numbers such that u + v = u' + v' = t and p = u/t, p' = u'/t and 
|p — 1/21 < |p' — 1/21. This means that (p, 1 — p) is closer to the uniform Bernoulli distribution compared to 
(p', 1 — p'). Then it is easy to check that 

(■ u , ) 1 ~ s + (n') 1 " 5 < u 1 ~ s + v 1 - 3 . (377) 


Now denoting ^ + <5i := ^ ae /-i(j) -Pa( a), jj-fa '■= for positive numbers and 5 2 (positive by 

the above construction of the sets / -1 (i) and / -1 (j)), and letting q := Pa(cl 2 ) < <Ji, t := (jj + <5i) + (jj — $ 2 ), we 
find that j(jj + 5\ — q, jj — 5 2 + q) is closer to the uniform Bernoulli distribution compared to \{jj + <5i, jj — 5 2 ). 
Using inequality (|377|>, we find that 



Since / 1 (k) = f 1 (k) for all k ^ {i,j}, ( |378| ) implies that 


E( E Pa ^) 

i V ae/- 1 F) 7 

contradicting the optimality of /. 



(378) 


(379) 


B. Proof of ( fTT3| > 

Proof: Here, we employ the following expression for C[_ s {A\E\Pae)'- 


e -f!r.Ctl-M\ E \PAx) 



^2 p E{e) 




(380) 


To minimize 


*-fb-Ct-sU{A)\E\PA E ) 


e it is enough to min imize p A\E=e(a)) * for each e. 

Fortunately, the discussion in the proof in Appendix E-A (and, in particular, the bound ( |11 ![ )) shows that this 
value is upper bounded by 


E P A\E{ a \ e ) 1 S + P A\E=e{ a '■ P A\E( a \ e ) < 

a:P A \E= B (a)>^ 


f) 

Mi 


1 — S 


M s 


(381) 
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Thus, 


e -T^ s CLs(f(A)\E\P AB ) 

< : P M E,e(a) > {j}c-M- 

+ P A\E=e{° '■ P A\E( a \e) < M A 

= ^2 p E(e)(p A \E=e{a ■ PA\E=e(a ) > Jj}c~ s 
e ' 

{ ,C ~] 1 — 1-8 

a : P A \E(a\e ) < — j J . 


Hence, we obtain (113) as desired. 


(382) 


(383) 


(384) 


C. Proofs of ( |114| ) and ( | 115 > 

Proof: The proofs of these bounds are similar to those of ( | 1111 ) and ( 112 ) in Appendix |E-A and thus are 


omitted. 


D. Proof of ( |1 16[ ) 


Proof: The proof of this bound is similar to that of ( |1 13| ) in Appendix |E-B| and is thus omitted. 


Appendix F 
Proof of Lemma [6] 

A. Proof of ( 117[ ) 

Proof: The proof of this bound is similar to the proof of (109) which is presented in Appendix D-A We 
provide the details here. 

When P 4 | E (a|e) < we have P A \ E (a\e) l ~ s M s < P A \ E (a\e)c~ s . Thus, starting from ( |359[ ), we have 

1 _ 2e -*C , i-.(/(A)|B|P Jts ) 


> 1 - 2c -s p E(e) J2 PA\ E (a\e) 1 - s M s 

e a:P AlE (a\e)>^ 

- 2 • 2^s^(l - s)P A E[{a,e ) : P A \E{a\e) < jj] 

> 1 - 2 c~ s P A E^{a,e) : P A \E{a\e) > 

- 2c- 2^s^(l - s)P AE {(a,e) : P A \ E (a\e) < ^}- 


This completes the proof of (117). 


(385) 


(386) 
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